Chunks chart
Admin panel
You can monitor the state of all chunks in the cluster on the Chunks chart. Chunks can be in the following states:
- Healthy
- Number and percentage of chunks that have enough active replicas. The normal state of chunks.
- Offline
-
Number and percentage of chunks all replicas of which are offline. Such chunks are completely inaccessible for the cluster and cannot be replicated, read from or written to. All requests to an offline chunk are frozen until a CS that stores that chunk’s replica goes online.
Get offline chunk servers back online as fast as possible, to avoid losing data.
- Blocked
-
Number and percentage of chunks that have fewer active replicas than the set minimum amount. Write requests to a blocked chunk are frozen until it has at least the set minimum amount of replicas. Read requests to blocked chunks are allowed, however, as they still have some active replicas left. Blocked chunks have a higher replication priority than degraded chunks.
Having blocked chunks in the cluster increases the risk of losing data, so postpone any maintenance on working cluster nodes and get offline chunk servers back online as fast as possible.
- Degraded
- Number and percentage of chunks whose active replicas are few, but not below the set minimum. Such chunks can be read from and written to. However, in the latter case a degraded chunk becomes urgent.
Healthy chunks are highlighted on the scale in green, offline in red, blocked in yellow, and degraded in grey. For example:
The Replication section shows the information about replication activity in the cluster.
Command-line interface
Use the following command:
vinfra cluster overview
For example, to view the information about chunks in the cluster cluster1, take a look at these lines from the command output:
+-------------------+-------------------------+ | Field | Value | +-------------------+-------------------------+ | ... | ... | | chunks | blocked: 0 | | | degraded: 0 | | | deleting: 0 | | | healthy: 153 | | | offline: 0 | | | overcommitted: 0 | | | pending: null | | | replicating: 0 | | | standby: null | | | total: 153 | | | unique: 0 | | | urgent: 0 | | | void: 0 | | ... | ... | +-------------------+-------------------------+
blocked-
Number of chunks that have fewer active replicas than the set minimum amount. Write requests to a blocked chunk are frozen until it has at least the set minimum amount of replicas. Read requests to blocked chunks are allowed, however, as they still have some active replicas left. Blocked chunks have a higher replication priority than degraded chunks.
Having blocked chunks in the cluster increases the risk of losing data, so postpone any maintenance on working cluster nodes and get offline chunk servers back online as fast as possible.
degraded- Number of chunks whose active replicas are few, but not below the set minimum. Such chunks can be read from and written to. However, in the latter case, a degraded chunk becomes urgent.
deleting- Number of chunks queued for deletion.
healthy- Number of chunks that have enough active replicas. The normal state of chunks.
offline-
Number of chunks all replicas of which are offline. Such chunks are completely inaccessible for the cluster and cannot be replicated, read from or written to. All requests to an offline chunk are frozen until a CS that stores that chunk’s replica goes online.
Get offline chunk servers back online as fast as possible, to avoid losing data.
overcommitted- Number of chunks that have more replicas than normal. Usually these chunks appear after the normal number of replicas has been lowered or a lot of data has been deleted. Extra replicas are eventually dropped, however, this process may slow down during replication.
pending- Number of chunks that must be replicated immediately. For a write request from client to a chunk to complete, the chunk must have at least the set minimum amount of replicas. If it does not, the chunk is blocked and the write request cannot be completed. As blocked chunks must be replicated as soon as possible, the cluster places them in a special high-priority replication queue and reports them as pending.
replicating- Number and percentage of chunks which are being replicated. Write operations on such chunks are frozen until replication ends.
standby- Number of chunks that have one or more replicas in the standby state. A replica is marked standby if it has been inactive for no more than 5 minutes.
total- Total number of all chunks in the storage cluster.
unique- Number of chunks that do not have replicas.
urgent- Number and percentage of chunks which are degraded and have non-identical replicas. Replicas of a degraded chunk may become non-identical if some of them are not accessible during a write operation. As a result, some replicas happen to have the new data while some still have the old data. The latter are dropped by the cluster as fast as possible. Urgent chunks do not affect information integrity as the actual data is stored in at least the set minimum amount of replicas.
void- Number and percentage of chunks that have been allocated but never used yet. Such chunks contain no data. It is normal to have some void chunks in the cluster.
See also