Compute alerts

Based on the metrics described in Compute metrics, the compute alerts are generated and displayed in the admin panel.

Compute service alerts

OpenStack Cinder API is down

  1. Ensure that the cinder_api container is up on the management node by running:

    # docker ps --all | grep cinder_api

  2. If the container is down, start it by running:

    # docker start cinder_api
  3. Check the service log at /var/log/hci/cinder/cinder-api.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Cinder Scheduler is down

  1. Ensure that the cinder_scheduler container is up on the specified node by running:

    # docker ps --all | grep cinder_scheduler

  2. If the container is down, start it by running:

    # docker start cinder_scheduler
  3. Check the service log at /var/log/hci/cinder/cinder-scheduler.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Cinder Volume agent is down

  1. Ensure that the cinder_volume container is up on the specified node by running:

    # docker ps --all | grep cinder_volume

  2. If the container is down, start it by running:

    # docker start cinder_volume
  3. Check the service log at /var/log/hci/cinder/cinder-volume.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Glance API is down

  1. Ensure that the glance_api container is up on the management node by running:

    # docker ps --all | grep glance_api

  2. If the container is down, start it by running:

    # docker start glance_api
  3. Check the service log at /var/log/hci/glance/glance-api.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Heat API is down

  1. Ensure that the heat_api container is up on the management node by running:

    # docker ps --all | grep heat_api

  2. If the container is down, start it by running:

    # docker start heat_api
  3. Check the service log at /var/log/hci/heat/heat-api.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Magnum API is down

  1. Ensure that the magnum_api container is up on the management node by running:

    # docker ps --all | grep magnum_api

  2. If the container is down, start it by running:

    # docker start magnum_api
  3. Check the service log at /var/log/hci/magnum/magnum-api.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Neutron API is down

  1. Ensure that the neutron_server container is up on the management node by running:

    # docker ps --all | grep neutron_server

  2. If the container is down, start it by running:

    # docker start neutron_server
  3. Check the service log at /var/log/hci/neutron/neutron-server.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Neutron L3 agent is down

  1. Ensure that the neutron_l3_agent container is up on the specified node by running:

    # docker ps --all | grep neutron_l3_agent

  2. If the container is down, start it by running:

    # docker start neutron_l3_agent
  3. Check the service log at /var/log/hci/neutron/neutron-l3-agent.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Neutron OpenvSwitch agent is down

  1. Ensure that the neutron_openvswitch_agent container is up on the specified node by running:

    # docker ps --all | grep neutron_openvswitch_agent

  2. If the container is down, start it by running:

    # docker start neutron_openvswitch_agent
  3. Check the service log at /var/log/hci/neutron/neutron-openvswitch-agent.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Neutron Metadata agent is down

  1. Ensure that the neutron_metadata_agent container is up on the specified node by running:

    # docker ps --all | grep neutron_metadata_agent

  2. If the container is down, start it by running:

    # docker start neutron_metadata_agent
  3. Check the service log at /var/log/hci/neutron/neutron-metadata-agent.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Neutron DHCP agent is down

  1. Ensure that the neutron_dhcp_agent container is up on the specified node by running:

    # docker ps --all | grep neutron_dhcp_agent

  2. If the container is down, start it by running:

    # docker start neutron_dhcp_agent
  3. Check the service log at /var/log/hci/neutron/neutron-dhcp-agent.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Nova API is down

  1. Ensure that the nova_api container is up on the management node by running:

    # docker ps --all | grep nova_api

  2. If the container is down, start it by running:

    # docker start nova_api
  3. Check the service log at /var/log/hci/nova/nova-api.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Nova Compute is down

  1. Ensure that the nova_compute container is up on the specified node by running:

    # docker ps --all | grep nova_compute

  2. If the container is down, start it by running:

    # docker start nova_compute
  3. Check the service log at /var/log/hci/nova/nova-compute.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Nova Conductor is down

  1. Ensure that the nova_conductor container is up on the specified node by running:

    # docker ps --all | grep nova_conductor

  2. If the container is down, start it by running:

    # docker start nova_conductor
  3. Check the service log at /var/log/hci/nova/nova-conductor.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Nova Scheduler is down

  1. Ensure that the nova_scheduler container is up on the specified node by running:

    # docker ps --all | grep nova_scheduler

  2. If the container is down, start it by running:

    # docker start nova_scheduler
  3. Check the service log at /var/log/hci/nova/nova-scheduler.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Octavia API is down

  1. Ensure that the octavia_api container is up on the management node by running:

    # docker ps --all | grep octavia_api

  2. If the container is down, start it by running:

    # docker start octavia_api
  3. Check the service log at /var/log/hci/octavia/octavia-api.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

OpenStack Placement API is down

  1. Ensure that the placement_api container is up on the management node by running:

    # docker ps --all | grep placement_api

  2. If the container is down, start it by running:

    # docker start placement_api
  3. Check the service log at /var/log/hci/placement/placement-api.log.
  4. If you cannot troubleshoot the problem, contact the technical support team.

High request error rate for OpenStack API requests detected

  1. Check the status of the affected compute services.
  2. If some services are down, bring them up.
  3. If you cannot troubleshoot the problem, contact the technical support team.

Compute cluster alerts

Compute cluster has failed

  1. Go to the Monitoring > Dashboard screen, and then click Grafana dashboard.
  2. Open the Compute service status dashboard and find out the failed service.
  3. Depending on the service, follow the instructions from the Compute service alerts section.

Cluster is running out of vCPU resources

The compute cluster may soon experience the lack of vCPU resources that will lead to inability to accommodate new virtual machines. To avoid this, you can add more compute nodes or return to operation fenced nodes, if any.

Cluster is out of vCPU resources

The compute cluster will soon experience the lack of vCPU resources that will lead to inability to accommodate new virtual machines. To avoid this, you can add more compute nodes or return to operation fenced nodes, if any.

Cluster is running out of memory

The compute cluster may soon experience the lack of RAM resources that will lead to inability to accommodate new virtual machines. To avoid this, you can add more compute nodes or return to operation fenced nodes, if any.

Cluster is out of memory

The compute cluster will soon experience the lack of RAM resources that will lead to inability to accommodate new virtual machines. To avoid this, you can add more compute nodes or return to operation fenced nodes, if any.

Virtual machine error

  1. Examine the VM history in the History tab on the VM right pane and reset the VM state, as described in Troubleshooting virtual machines.
  2. If you cannot troubleshoot the problem, contact the technical support team.

Virtual machine state mismatch

Do not try to migrate the VM or reset its state. Contact the technical support team.

Volume attachment details mismatch

Do not try to migrate the VM or reset its state. Contact the technical support team.

Compute node alerts

Node is running out of vCPU resources

The compute node may soon experience the lack of vCPU resources that will lead to inability to accommodate new virtual machines. To avoid this, check the distribution of VMs in the compute cluster, and then migrate the VMs from the specified node to less loaded compute nodes.

Node is out of vCPU resources

The compute node will soon experience the lack of vCPU resources that will lead to inability to accommodate new virtual machines. To avoid this, check the distribution of VMs in the compute cluster, and then migrate the VMs from the specified node to less loaded compute nodes.

Node is running out of memory

The compute node may soon experience the lack of RAM resources that will lead to inability to accommodate new virtual machines. To avoid this, check the distribution of VMs in the compute cluster, and then migrate the VMs from the specified node to less loaded compute nodes.

Node is out of memory

The compute node will soon experience the lack of RAM resources that will lead to inability to accommodate new virtual machines. To avoid this, check the distribution of VMs in the compute cluster, and then migrate the VMs from the specified node to less loaded compute nodes.

Project quota alerts

Project is out of vCPU resources

The project will soon experience the lack of vCPU resources that will lead to inability to create new virtual machines. To avoid this, add more vCPUs to the project quota.

Project is out of memory

The project will soon experience the lack of RAM resources that will lead to inability to create new virtual machines. To avoid this, add more RAM to the project quota.

Project is out of floating IP addresses

The project will soon experience the lack of floating IP addresses that will lead to inability to assign them to virtual machines. To avoid this, add more floating IPs to the project quota.

Network is out of IP addresses

The network will soon experience the lack of IP addresses that will lead to inability to connect new virtual machines to this network. To avoid this, add more allocation pools to the network.

Project is out of storage policy space

The project will soon experience the lack of storage policy space that will lead to inability to create new compute volumes with this storage policy. To avoid this, add more storage space to the project quota.

Other alerts

Libvirt service is down

Start the libvirtd service on the specified node by running:

# systemctl start libvirtd.service

Docker service is down

Start the Docker service on the specified node by running:

# systemctl start docker.service

RabbitMQ node is down

Contact the technical support team.

RabbitMQ split brain detected

Contact the technical support team.

PostgreSQL database size is greater than 30 GB

PostgreSQL database "<name>" on node "<hostname>" is greater than 30 GB in size. Verify that deleted entries are archived or contact the technical support.

PostgreSQL database uses more than 50% of node root partition

PostgreSQL databases on node "<hostname>" with ID "<id>" use more than 50% of node root partition. Verify that deleted entries are archived or contact the technical support.

Related Topics Link IconWhat's next
Related Topics Link IconSee also