3.5. Managing Disk I/O Parameters

This section explains how to manage disk input and output (I/O) parameters in Virtuozzo Hybrid Server systems.

Note that I/O and IOPS limiting only works for supported I/O schedulers. The scheduler type is stored in /sys/block/<device>/queue/scheduler. For example:

# cat /sys/block/sda/queue/scheduler
noop deadline [cfq]

The I/O scheduler used is marked by square brackets. In this example, it is cfq.

The following I/O schedulers are supported:

  • CFQ. A separate block device line with counters is added to the iostat file. For example:

    # cat /proc/bc/100/iostat
    flush 100 . 0 0 0 0 0 7389 1893968 0 0
    fuse 100 . 0 0 0 0 0 0 0 0 0
    sda 100 . 0 0 0 9000 1843380 245216 55845488 245028 188
    
  • Deadline. The counters are added to the values in the flush line.

I/O and IOPS limiting does not work for devices with the noop I/O scheduler or without one (e.g., logical devices, CEPH RBD devices, and such).

3.5.1. Configuring Priority Levels for Virtual Machines and Containers

In Virtuozzo Hybrid Server, you can configure the disk I/O (input/output) priority level of virtual machines and containers. The higher the I/O priority level, the more time the virtual machine or container will get for its disk I/O activities as compared to the other virtual machines and containers on the hardware node. By default, any virtual machine or container on the hardware node has the I/O priority level set to 4. However, you can change the current I/O priority level in the range from 0 to 7 using the --ioprio option of the prlctl set command. For example, you can issue the following command to set the I/O priority of the container MyCT and the virtual machine MyVM to 6:

# prlctl set MyCT --ioprio 6
# prlctl set MyVM --ioprio 6

To check the I/O priority level currently applied to the container MyCT and the virtual machine MyVM, you can execute the following commands:

  • For container MyCT:

    # grep IOPRIO /etc/vz/conf/fbb30afa-e770-4081-9d9e-6b9c262eb091.conf
    IOPRIO="6"
    
  • For the virtual machine MyVM:

    # prlctl list MyVM --info | grep ioprio
    cpu cpus=2 VT-x accl=high mode=32 ioprio=6 iolimit='0'
    

3.5.2. Limiting Disk I/O Bandwidth

In Virtuozzo Hybrid Server, you can configure the bandwidth virtual machines and containers are allowed to use for their disk input and output (I/O) operations. Limiting the disk I/O bandwidth can help you prevent the situations when high disk activities in one virtual machine or container (generated, for example, by transferring huge amounts of data to/from the virtual machine or container) can slow down the performance of other virtual machines and containers on the hardware node.

By default, the I/O bandwidth limit for all newly created virtual machines and containers is set to 0, which means that no limits are applied. To limit the disk I/O bandwidth for a virtual machine or container, use the --iolimit option of the prlctl set command. For example, to set the I/O bandwidth limit for the container MyCT to 10 MB/s:

# prlctl set MyCT --iolimit 10

By default, the limit is set in megabytes per second. In addition, you can use the following suffixes to indicate other measurement units:

  • G sets the limit in gigabytes per second (1G).

  • K sets the limit in kilobytes per second (10K).

  • B sets the limit in bytes per second (10B).

Note

In the current version of Virtuozzo Hybrid Server, the maximum I/O bandwidth limit you can set for a virtual machine or container is 2 GB per second.

To check that the I/O speed limit has been successfully applied to the container MyCT, use the prlctl list command:

# prlctl list MyCT -o iolimit
IOLIMIT
10485760

At any time, you can remove the I/O bandwidth limit set for container MyCT by running this command:

# prlctl set MyCT --iolimit 0

3.5.3. Limiting the Number of I/O Operations per Second

In Virtuozzo Hybrid Server, you can limit the maximum number of disk input and output operations per second virtual machines and containers are allowed to perform (known as the IOPS limit). You may consider setting the IOPS limit for virtual machines and containers with high disk activities to ensure that they do not affect the performance of other virtual machines and containers on the Node.

Note

By default all I/O inside containers is cached and the direct access flag (O_DIRECT) is ignored when opening files. This significantly reduces the number of IOPS required for container workload and helps avoid I/O bottlenecks on the Node. For instructions on how to configure honoring of the O_DIRECT flag inside containers, see Setting the Direct Access Flag Inside Containers further.

By default, IOPS is not limited for newly created virtual machines and containers. To set the IOPS limit, you can use the --iopslimit option of the prlctl set command. For example, to allow the container MyCT and the virtual machine MyVM to perform no more than 100 disk I/O operations per second, you can run the following commands:

# prlctl set MyCT --iopslimit 100
# prlctl set MyVM --iopslimit 100

To ensure that the IOPS limit has been successfully applied, use the prlctl list -i command. For example:

# prlctl list MyCT -i | grep iopslimit
<...> iopslimit=100

At any time, you can remove the set IOPS limits by running this command:

# prlctl set MyCT --iopslimit 0
# prlctl set MyVM --iopslimit 0

3.5.3.1. Setting the Direct Access Flag Inside Containers

You can configure honoring of the O_DIRECT flag inside containers with the sysctl parameter fs.odirect_enable:

  • To ignore the O_DIRECT flag inside a container, set fs.odirect_enable to 0 in that container.

  • To honor the O_DIRECT flag inside the container, set fs.odirect_enable to 1 in that container.

  • To have a container inherit the setting from the hardware node, set fs.odirect_enable to 2 in that container (default value). On the hardware node, fs.odirect_enable is 0 by default.

    Note

    The fs.odirect_enable parameter on the Node only affects honoring of the O_DIRECT flag in containers and not on the Node itself where the O_DIRECT flag is always honored.

3.5.4. Viewing Disk I/O Statistics

In Virtuozzo Hybrid Server, you can view disk input and output (I/O) statistics for all processes on the host. To do this:

  1. Run the vztop utility.

  2. Press F2 or S to switch to the Setup menu.

  3. In the Setup column, choose Columns.

  4. In Available Columns, choose from the following parameters to add to the output (Active Columns):

    Parameter

    Description

    Column

    RBYTES

    Number of bytes read for the process.

    IO_RBYTES

    WBYTES

    Number of bytes written for the process.

    IO_WBYTES

    IO_READ_RATE

    Process read rate, in bytes per second.

    DISK READ

    IO_WRITE_RATE

    Process write rate, in bytes per second.

    DISK WRITE

    IO_RATE

    Process total I/O rate, in bytes per second.

    DISK R/W

    IO_PRIORITY

    Process I/O priority.

    IO

    To add a parameter, select it and press F5 or Enter. To remove a parameter from Active Columns, select it and press F9.

  5. When you finish managing columns, press F10 to save the changes and view the output.

3.5.5. Setting I/O Limits for Backup and Migration Operations

Backup and migration operations with containers and virtual machines can generate a high I/O load on the server, thus reducing the performance of other virtual environments or the server itself. You can avoid such situations by setting I/O limits for these operations.

To set an I/O limit, do the following:

  1. In the /etc/vz/vz.conf global configuration file, locate the following section:

    # VZ Tools limits
    <...>
    # Uncomment next line to specify required disk IO bandwidth in Bps (10485760 - 10MBps)
    # VZ_TOOLS_IOLIMIT=10485760
    
  2. Uncomment the VZ_TOOLS_IOLIMIT parameter, and set the I/O limit for backup and migration operations in bytes per second.

  3. Save the file.

When setting I/O limits, pay attention to the following:

  • VZ_TOOLS_IOLIMIT is a global parameter that has effect on all virtual environments on the server.

  • The VZ_TOOLS_IOLIMIT parameter controls the I/O load only for backup and migration operations. Restore operations are not limited.

  • Simultaneous operations do not share the limit and are limited separately.

  • For migration, only the limit set on the source server applies, while the limit set on the destination server is ignored.

3.5.6. Improving Disk I/O Performance for Virtual Machines

In case of VM virtual disks mapped to QCOW2 disk images, the disk space is only allocated when it is actually needed by the VM and the cache size depends on how much space is allocated. In Virtuozzo Hybrid Server, you can significantly improve disk I/O performance by means of cache policies applied to VM disks based on their allocated space size.

The default cache policy is used for typical VM workloads and disk sizes up to 1TB. If a guest needs continuous access to an entire disk that is larger than 1TB, the default cache size will not be sufficient and will reduce disk I/O perfomance. In this case, you can increase cache size to speed up disk access as follows:

  1. In the /etc/libvirt/qemu.conf configuration file, uncomment and set the qcow2_cache_policy parameter to disk.

  2. Restart the libvirtd daemon to apply the changes.

    # systemctl restart libvirtd
    

The disk policy will provide a cache large enough to handle the entire disk workload.