Monitoring infrastructure nodes

Nodes added to the infrastructure are listed on the Infrastructure > Nodes screen. If the storage cluster has not been created yet, you will only see nodes with the Unassigned status.

A node can have one of the following statuses:

Healthy: All the storage services on the node are running.
Unhealthy: One or more storage services on the node have failed.
In maintenance: The node is in maintenance mode. It does not participate in new chunk allocation.
Recovering: The configuration of deployed services and infrastructure is being recovered on the node.
Recovery failed: The recovery process has failed on the node.
In progress: The node is deploying, entering, or exiting maintenance. Nodes in this state cannot be managed.
Unassigned: The node is not assigned to a storage cluster.

To view the node details

Admin panel

Open the Infrastructure > Nodes screen and click the required node line. On the right pane, the Overview tab includes the node details, such as the node ID and hostname, its status and location, assigned IP addresses, deployed services, and the number of disks and network interfaces.

Command-line interface

Use the following command:

vinfra node show <node>

<node>: Node ID or hostname

For example, to view the details of the node node003, run:

# vinfra node show node003
+---------------+--------------------------------------+
| Field         | Value                                |
+---------------+--------------------------------------+
| has_kvm       | True                                 |
| host          | node003.vstoragedomain               |
| id            | c4d14337-0863-4a67-9dbd-f19c3e49e114 |
| is_assigned   | True                                 |
| is_in_ha      | False                                |
| is_installing | False                                |
| is_online     | True                                 |
| is_primary    | False                                |
| is_virt       | False                                |
| maintenance   |                                      |
| orig_hostname | node003                              |
| roles         | cses:                                |
|               |   active: 4                          |
|               |   being_released: 0                  |
|               | mdses:                               |
|               |   avail: 1                           |
|               |   being_released: 0                  |
|               |   is_master: true                    |
| tasks         |                                      |
+---------------+--------------------------------------+

In the command output, the node details include the node ID, hostname, the number and status of storage services. Also, you can check whether the node is joined to the storage cluster and HA configuration, and if it is online or in the maintenance mode.

To view more details about infrastructure nodes

Go to the Monitoring > Dashboard screen, and then click Grafana dashboard. A separate browser tab will open with preconfigured Grafana dashboards.

On the Hardware nodes overview dashboard, you can monitor the following:

Node availability and load average
Amount of available RAM and swap usage
Normalized CPU usage
Amount of free space on the root partition and storage disks
Disk bandwidth, IOPS, and average disk utilization
Network bandwidth, packet rate, and errors

The displayed data can be filtered per node or network.

The Utilization dashboard shows vCPU, vRAM, and storage space usage in the entire cluster and on each node separately. To see usage dynamics, you can change the default time range of seven days to 30 (one month) or 90 (three months).

Open the Process details dashboard, to see a list of processes on your infrastructure nodes. The charts on this dashboard show CPU and memory usage for each process. The displayed data can be filtered per node or process group.