Monitoring chunk servers
By monitoring chunk servers, you can keep track of the disk space available in the storage cluster. To monitor chunk servers, use the vstorage -c <cluster_name> top
command. For example:
The command above shows detailed information about the stor1
cluster. The monitoring parameters for chunk servers (highlighted in red) are the following:
- CSID
- Chunk server identifier (ID).
- STATUS
-
Chunk server status:
- active
- The chunk server is up and running.
- failed
- The chunk server process is running but a problem has occured with the CS disk.
- inactive
- The chunk server is temporarily unavailable. A chunk server is marked as inactive during its first 5 minutes of inactivity.
- offline
- The chunk server is inactive for more than 5 minutes. After the chunk server goes offline, the cluster starts replicating data to restore the chunks that were stored on the affected chunk server.
- dropped
- The chunk server was removed by the administrator.
- maintenance
- The node that the chunk server is located on is in maintenance.
- ill
- The chunk server experiences slowdown and degrades the cluster performance. The chunk server is isolated from the cluster I/O.
- SPACE
- Total amount of disk space on the chunk server.
- AVAIL
- Available disk space on the chunk server.
- REPLICAS
- Number of replicas stored on the chunk server.
- UNIQUE
- Number of chunks that do not have replicas.
- IOWAIT
- Percentage of time spent waiting for I/O operations being served.
- IOLAT
- Average/maximum time, in milliseconds, the client needed to complete a single IO operation during the last 20 seconds.
- QDEPTH
- Average chunk server I/O queue depth.
- HOST
- Chunk server hostname or IP address.
- FLAGS
The following flags may be shown for active chunk servers:
- J
- The CS uses a write journal.
- C
- Checksumming is enabled for the CS. Checksumming lets you know when a third party changes the data on the disk.
- D
- Direct I/O, the normal state for a CS without a write journal.
- c
- The chunk server’s write journal is clean, there is nothing to commit from the write journaling SSD to the HDD where the CS is located.