3.5. Viewing Alerts and Audit Log and Sending E-mail Notifications

This section describes Virtuozzo Hybrid Infrastructure alerts and audit log and how to send out e-mail notifications about alerts, warnings, and errors.

3.5.1. Viewing Alerts

The Alerts tab lists all the alerts logged by Virtuozzo Hybrid Infrastructure. An alert is generated and logged each time one of the following conditions is met or events happen:

  • a critical issue has happened with a cluster, its components (CS, MDS), disks, nodes, or services;
  • cluster requires configuration or more resources to build or restore its health;
  • network requires configuration or is experiencing issues that may affect performance;
  • license has expired;
  • cluster is about to or has run out of available space.
../_images/stor_image32_1_vz.png

To view the details of an alert, select it on the MONITORING > Alerts screen and click Details in the menu on the right.

Alerts can be ignored (deleted from the alerts list) or postponed for several hours. Postponed alerts reappear in the list after some time.

To ignore or postpone an alert, select it and click the corresponding button.

The following alerts are generated and displayed in the admin panel:

Table 3.5.1.2 Alerts
Title Message Severity
License alerts
License is not loaded License is not installed. warning
License expired The license of cluster “<cluster_name>” has expired. Сontact your reseller to update your license immediately! critical
Cluster alerts
Cluster is out of space Cluster has just <free_space> TB (<free_space_in_percent>%) of physical storage space left. You may want to free some space or add more storage capacity. warning
Сluster “<cluster_name>” has run out of storage space allowed by license. No more data can be written. Please contact your reseller to update your license immediately! warning
Not enough cluster nodes Cluster “<cluster_name>” has only {1,2} node(s) instead of the recommended minimum of 3. Add {2,1} or more nodes to the cluster. warning
High availability for the admin panel must be configured Configure high availability for the admin panel in SETTINGS > Management node. Otherwise the admin panel will be a single point of failure. critical
The last management node backup has failed, does not exist, or is too old Management node backup is older than <number_of_days> days. critical
Management node backup does not exist. critical
Metadata service alerts
Not enough metadata disks Cluster “<cluster_name>” has only one MDS. There is only one disk with the metadata role at the moment. Losing this disk will completely destroy all cluster data irrespective of the redundancy schema. warning
Cluster “<cluster_name>” requires more disks with the metadata role. Losing one more MDS will halt cluster operation. warning
Configuration warning Node “<hostname>” has more than one metadata service located on it. It is recommended to have only one metadata service per node. Delete the extra metadata service(s) from this node and create them on other nodes instead. warning
Service failed Metadata service #<id> is in the “<status>” state. Node: <hostname>. Disk: <disk_name>. Disk serial: <disk_serial>. warning
Metadata disk is out of space Metadata disk on node “<hostname>” is running out of space. warning
Chunk service alerts
Not enough disks with storage role Cluster “<cluster_name>” has no disks with the storage role. warning
Cluster “<cluster_name>” has too few available CSes. warning
Service failed Storage service #<id> is in the “<status>” state. Node: <hostname>. Disk: <disk_name>. Disk serial: <disk_serial>. warning
CS configuration is not optimal CS#<cs_id> on tier <tier> has incorrect journalling settings. warning
Encryption is disabled for CS#<cs_id> on tier <tier> but is enabled for other CSes on the same tier. warning
Node alerts
Node is offline Node “<hostname>” is offline. warning
Software updates exist Software updates are available for the node “<hostname>”. warning
Kernel is outdated Node “<hostname>” is not running the latest kernel. warning
OOM killer triggered OOM killer has been triggered on node “<hostname>”. warning
Time is not synced Time on node “<hostname>” is out of sync with the management node. warning
Disk alerts
S.M.A.R.T. warning Disk “<disk_name>”(<serial>) on node “<hostname>” has failed a S.M.A.R.T. check. critical
Disk error Disk “<disk_name>” (<serial>) failed on node “<hostname>”. critical
Disk is out of space Root partition on node “<hostname>” is running out of space. warning
Disk write cache disabled Disk write cache is disabled for disk “<disk_name>” on node “<hostname>”. warning
Disk write cache status unknown Cannot determine the status of write cache for disk “<disk_name>” on node “<hostname>”. warning
Network alerts
No internet connection No Internet connection on the node “<hostname>”. warning
Network warning Network interface “<iface_name>” has incorrect settings: <duplex> duplex and <speed> speed. warning
Network interface “<iface_name>” on node “<hostname>” is missing important features (or has them disabled): “<feature_name>”. warning
Network interface “<iface_name>” on node “<hostname>” is not in the full duplex mode. warning
Network interface “<iface_name>” on node “<hostname>” has speed lower than the minimally required 1 Gbps. warning
Network interface “<iface_name>” on node “<hostname>” has an undefined speed. warning
Other alerts
Compute cluster is failed Compute cluster has failed. Management of virtual machines may be unavailable. critical
Redundancy warning iSCSI LUN <lun_id> of target group “<target_group>” is set to failure domain “disk” even though <number_of_nodes> nodes are available. It is recommended to set the failure domain to “host” so that the LUN can survive host failures in addition to disk failures. warning
S3 is set to failure domain “disk” even though <number_of_nodes> nodes are available. It is recommended to set the failure domain to “host” so that S3 can survive host failures in addition to disk failures. warning
Certificate expiration Acronis Backup Gateway certificate has expired or will expire soon. warning
iSCSI major upgrade failed iSCSI major upgrade failed. Will be retried… critical

3.5.2. Viewing Audit Log

The MONITORING > Audit log screen lists all management operations performed by users and their activity events.

../_images/stor_image32_2_vz.png

To view detailed information on a log entry, select it and click Show more details.

3.5.3. Sending E-mail Notifications

Virtuozzo Hybrid Infrastructure can send automatic e-mail notifications about errors, warnings, and alerts.

To set up e-mail notifications, do the following:

  1. On the SETTINGS > Advanced settings > NOTIFICATIONS tab, specify the following information:

    1. In the From and Sender name fields, the notification sender’s e-mail and name.

    2. In the To field, one or more notification recipient e-mails, one per line.

    3. In the User account and User password fields, the credentials of the notification sender registered on the SMTP server.

    4. In the SMTP server field, the DNS name of the SMTP server, either public (e.g., smtp.gmail.com) or the one in your organization.

      The management node must be able to access the SMTP server.

    5. If required, a custom SMTP port the server uses.

    6. In the Security field, the security protocol of the SMTP server.

    ../_images/stor_image32_3_vz.png
  2. Tick the checkboxes for alerts you want to be notified about.

  3. Click Save.

To send a test e-mail, specify your e-mail registered on the SMTP server in both the From and To fields and click Test.