3.5. Managing Disk I/O Parameters¶
This section explains how to manage disk input and output (I/O) parameters in Virtuozzo Hybrid Server systems.
Note that I/O and IOPS limiting only works for supported I/O schedulers. The scheduler type is stored in
/sys/block/<device>/queue/scheduler. For example:
# cat /sys/block/sda/queue/scheduler noop deadline [cfq]
The I/O scheduler used is marked by square brackets. In this example, it is cfq.
The following I/O schedulers are supported:
CFQ. A separate block device line with counters is added to the
iostatfile. For example:
# cat /proc/bc/100/iostat flush 100 . 0 0 0 0 0 7389 1893968 0 0 fuse 100 . 0 0 0 0 0 0 0 0 0 sda 100 . 0 0 0 9000 1843380 245216 55845488 245028 188
Deadline. The counters are added to the values in the
I/O and IOPS limiting does not work for devices with the noop I/O scheduler or without one (e.g., logical devices, CEPH RBD devices, and such).
3.5.1. Configuring Priority Levels for Virtual Machines and Containers¶
In Virtuozzo Hybrid Server, you can configure the disk I/O (input/output) priority level of virtual machines and containers. The higher the I/O priority level, the more time the virtual machine or container will get for its disk I/O activities as compared to the other virtual machines and containers on the hardware node. By default, any virtual machine or container on the hardware node has the I/O priority level set to 4. However, you can change the current I/O priority level in the range from 0 to 7 using the
--ioprio option of the
prlctl set command. For example, you can issue the following command to set the I/O priority of the container
MyCT and the virtual machine
MyVM to 6:
# prlctl set MyCT --ioprio 6 # prlctl set MyVM --ioprio 6
To check the I/O priority level currently applied to the container
MyCT and the virtual machine
MyVM, you can execute the following commands:
# grep IOPRIO /etc/vz/conf/fbb30afa-e770-4081-9d9e-6b9c262eb091.conf IOPRIO="6"
For the virtual machine
# prlctl list MyVM --info | grep ioprio cpu cpus=2 VT-x accl=high mode=32 ioprio=6 iolimit='0'
3.5.2. Limiting Disk I/O Bandwidth¶
In Virtuozzo Hybrid Server, you can configure the bandwidth virtual machines and containers are allowed to use for their disk input and output (I/O) operations. Limiting the disk I/O bandwidth can help you prevent the situations when high disk activities in one virtual machine or container (generated, for example, by transferring huge amounts of data to/from the virtual machine or container) can slow down the performance of other virtual machines and containers on the hardware node.
By default, the I/O bandwidth limit for all newly created virtual machines and containers is set to 0, which means that no limits are applied. To limit the disk I/O bandwidth for a virtual machine or container, use the
--iolimit option of the
prlctl set command. For example, to set the I/O bandwidth limit for the container
MyCT to 10 MB/s:
# prlctl set MyCT --iolimit 10
By default, the limit is set in megabytes per second. In addition, you can use the following suffixes to indicate other measurement units:
Gsets the limit in gigabytes per second (1G).
Ksets the limit in kilobytes per second (10K).
Bsets the limit in bytes per second (10B).
In the current version of Virtuozzo Hybrid Server, the maximum I/O bandwidth limit you can set for a virtual machine or container is 2 GB per second.
To check that the I/O speed limit has been successfully applied to the container
MyCT, use the
prlctl list command:
# prlctl list MyCT -o iolimit IOLIMIT 10485760
At any time, you can remove the I/O bandwidth limit set for container
MyCT by running this command:
# prlctl set MyCT --iolimit 0
3.5.3. Limiting the Number of I/O Operations per Second¶
In Virtuozzo Hybrid Server, you can limit the maximum number of disk input and output operations per second virtual machines and containers are allowed to perform (known as the IOPS limit). You may consider setting the IOPS limit for virtual machines and containers with high disk activities to ensure that they do not affect the performance of other virtual machines and containers on the Node.
By default all I/O inside containers is cached and the direct access flag (
O_DIRECT) is ignored when opening files. This significantly reduces the number of IOPS required for container workload and helps avoid I/O bottlenecks on the Node. For instructions on how to configure honoring of the
O_DIRECT flag inside containers, see Setting the Direct Access Flag Inside Containers further.
By default, IOPS is not limited for newly created virtual machines and containers. To set the IOPS limit, you can use the
--iopslimit option of the
prlctl set command. For example, to allow the container
MyCT and the virtual machine
MyVM to perform no more than 100 disk I/O operations per second, you can run the following commands:
# prlctl set MyCT --iopslimit 100 # prlctl set MyVM --iopslimit 100
To ensure that the IOPS limit has been successfully applied, use the
prlctl list -i command. For example:
# prlctl list MyCT -i | grep iopslimit <...> iopslimit=100
At any time, you can remove the set IOPS limits by running this command:
# prlctl set MyCT --iopslimit 0 # prlctl set MyVM --iopslimit 0
188.8.131.52. Setting the Direct Access Flag Inside Containers¶
You can configure honoring of the
O_DIRECT flag inside containers with the
To ignore the
O_DIRECTflag inside a container, set
0in that container.
To honor the
O_DIRECTflag inside the container, set
1in that container.
To have a container inherit the setting from the hardware node, set
2in that container (default value). On the hardware node,
fs.odirect_enableparameter on the Node only affects honoring of the
O_DIRECTflag in containers and not on the Node itself where the
O_DIRECTflag is always honored.
3.5.4. Viewing Disk I/O Statistics¶
In Virtuozzo Hybrid Server, you can view disk input and output (I/O) statistics for all processes on the host. To do this:
Press F2 or S to switch to the Setup menu.
In the Setup column, choose Columns.
In Available Columns, choose from the following parameters to add to the output (Active Columns):
Number of bytes read for the process.
Number of bytes written for the process.
Process read rate, in bytes per second.
Process write rate, in bytes per second.
Process total I/O rate, in bytes per second.
Process I/O priority.
To add a parameter, select it and press F5 or Enter. To remove a parameter from Active Columns, select it and press F9.
When you finish managing columns, press F10 to save the changes and view the output.
3.5.5. Setting I/O Limits for Backup and Migration Operations¶
Backup and migration operations with containers and virtual machines can generate a high I/O load on the server, thus reducing the performance of other virtual environments or the server itself. You can avoid such situations by setting I/O limits for these operations.
To set an I/O limit, do the following:
/etc/vz/vz.confglobal configuration file, locate the following section:
# VZ Tools limits <...> # Uncomment next line to specify required disk IO bandwidth in Bps (10485760 - 10MBps) # VZ_TOOLS_IOLIMIT=10485760
VZ_TOOLS_IOLIMITparameter, and set the I/O limit for backup and migration operations in bytes per second.
Save the file.
When setting I/O limits, pay attention to the following:
VZ_TOOLS_IOLIMITis a global parameter that has effect on all virtual environments on the server.
VZ_TOOLS_IOLIMITparameter controls the I/O load only for backup and migration operations. Restore operations are not limited.
Simultaneous operations do not share the limit and are limited separately.
For migration, only the limit set on the source server applies, while the limit set on the destination server is ignored.
3.5.6. Improving Disk I/O Performance for Virtual Machines¶
In case of VM virtual disks mapped to QCOW2 disk images, the disk space is only allocated when it is actually needed by the VM and the cache size depends on how much space is allocated. In Virtuozzo Hybrid Server, you can significantly improve disk I/O performance by means of cache policies applied to VM disks based on their allocated space size.
The default cache policy is used for typical VM workloads and disk sizes up to 1TB. If a guest needs continuous access to an entire disk that is larger than 1TB, the default cache size will not be sufficient and will reduce disk I/O perfomance. In this case, you can increase cache size to speed up disk access as follows:
/etc/libvirt/qemu.confconfiguration file, uncomment and set the
libvirtddaemon to apply the changes.
# systemctl restart libvirtd
disk policy will provide a cache large enough to handle the entire disk workload.