3.5. Managing Disk I/O Parameters¶
This section explains how to manage disk input and output (I/O) parameters in Virtuozzo Hybrid Server systems.
Note that I/O and IOPS limiting only works for supported I/O schedulers. The scheduler type is stored in /sys/block/<device>/queue/scheduler
. For example:
# cat /sys/block/sda/queue/scheduler
noop deadline [cfq]
The I/O scheduler used is marked by square brackets. In this example, it is cfq.
The following I/O schedulers are supported:
CFQ. A separate block device line with counters is added to the
iostat
file. For example:# cat /proc/bc/100/iostat flush 100 . 0 0 0 0 0 7389 1893968 0 0 fuse 100 . 0 0 0 0 0 0 0 0 0 sda 100 . 0 0 0 9000 1843380 245216 55845488 245028 188
Deadline. The counters are added to the values in the
flush
line.
I/O and IOPS limiting does not work for devices with the noop I/O scheduler or without one (e.g., logical devices, CEPH RBD devices, and such).
3.5.1. Configuring Priority Levels for Virtual Machines and Containers¶
In Virtuozzo Hybrid Server, you can configure the disk I/O (input/output) priority level of virtual machines and containers. The higher the I/O priority level, the more time the virtual machine or container will get for its disk I/O activities as compared to the other virtual machines and containers on the hardware node. By default, any virtual machine or container on the hardware node has the I/O priority level set to 4. However, you can change the current I/O priority level in the range from 0 to 7 using the --ioprio
option of the prlctl set
command. For example, you can issue the following command to set the I/O priority of the container MyCT
and the virtual machine MyVM
to 6:
# prlctl set MyCT --ioprio 6
# prlctl set MyVM --ioprio 6
To check the I/O priority level currently applied to the container MyCT
and the virtual machine MyVM
, you can execute the following commands:
For container
MyCT
:# grep IOPRIO /etc/vz/conf/fbb30afa-e770-4081-9d9e-6b9c262eb091.conf IOPRIO="6"
For the virtual machine
MyVM
:# prlctl list MyVM --info | grep ioprio cpu cpus=2 VT-x accl=high mode=32 ioprio=6 iolimit='0'
3.5.2. Limiting Disk I/O Bandwidth¶
In Virtuozzo Hybrid Server, you can configure the bandwidth virtual machines and containers are allowed to use for their disk input and output (I/O) operations. Limiting the disk I/O bandwidth can help you prevent the situations when high disk activities in one virtual machine or container (generated, for example, by transferring huge amounts of data to/from the virtual machine or container) can slow down the performance of other virtual machines and containers on the hardware node.
By default, the I/O bandwidth limit for all newly created virtual machines and containers is set to 0, which means that no limits are applied. To limit the disk I/O bandwidth for a virtual machine or container, use the --iolimit
option of the prlctl set
command. For example, to set the I/O bandwidth limit for the container MyCT
to 10 MB/s:
# prlctl set MyCT --iolimit 10
By default, the limit is set in megabytes per second. In addition, you can use the following suffixes to indicate other measurement units:
G
sets the limit in gigabytes per second (1G).K
sets the limit in kilobytes per second (10K).B
sets the limit in bytes per second (10B).
Note
In the current version of Virtuozzo Hybrid Server, the maximum I/O bandwidth limit you can set for a virtual machine or container is 2 GB per second.
To check that the I/O speed limit has been successfully applied to the container MyCT
, use the prlctl list
command:
# prlctl list MyCT -o iolimit
IOLIMIT
10485760
At any time, you can remove the I/O bandwidth limit set for container MyCT
by running this command:
# prlctl set MyCT --iolimit 0
3.5.3. Limiting the Number of I/O Operations per Second¶
In Virtuozzo Hybrid Server, you can limit the maximum number of disk input and output operations per second virtual machines and containers are allowed to perform (known as the IOPS limit). You may consider setting the IOPS limit for virtual machines and containers with high disk activities to ensure that they do not affect the performance of other virtual machines and containers on the Node.
Note
By default all I/O inside containers is cached and the direct access flag (O_DIRECT
) is ignored when opening files. This significantly reduces the number of IOPS required for container workload and helps avoid I/O bottlenecks on the Node. For instructions on how to configure honoring of the O_DIRECT
flag inside containers, see Setting the Direct Access Flag Inside Containers further.
By default, IOPS is not limited for newly created virtual machines and containers. To set the IOPS limit, you can use the --iopslimit
option of the prlctl set
command. For example, to allow the container MyCT
and the virtual machine MyVM
to perform no more than 100 disk I/O operations per second, you can run the following commands:
# prlctl set MyCT --iopslimit 100
# prlctl set MyVM --iopslimit 100
To ensure that the IOPS limit has been successfully applied, use the prlctl list -i
command. For example:
# prlctl list MyCT -i | grep iopslimit
<...> iopslimit=100
At any time, you can remove the set IOPS limits by running this command:
# prlctl set MyCT --iopslimit 0
# prlctl set MyVM --iopslimit 0
3.5.3.1. Setting the Direct Access Flag Inside Containers¶
You can configure honoring of the O_DIRECT
flag inside containers with the sysctl
parameter fs.odirect_enable
:
To ignore the
O_DIRECT
flag inside a container, setfs.odirect_enable
to0
in that container.To honor the
O_DIRECT
flag inside the container, setfs.odirect_enable
to1
in that container.To have a container inherit the setting from the hardware node, set
fs.odirect_enable
to2
in that container (default value). On the hardware node,fs.odirect_enable
is0
by default.Note
The
fs.odirect_enable
parameter on the Node only affects honoring of theO_DIRECT
flag in containers and not on the Node itself where theO_DIRECT
flag is always honored.
3.5.4. Viewing Disk I/O Statistics¶
In Virtuozzo Hybrid Server, you can view disk input and output (I/O) statistics for all processes on the host. To do this:
Run the
vztop
utility.Press F2 or S to switch to the Setup menu.
In the Setup column, choose Columns.
In Available Columns, choose from the following parameters to add to the output (Active Columns):
Parameter
Description
Column
RBYTES
Number of bytes read for the process.
IO_RBYTES
WBYTES
Number of bytes written for the process.
IO_WBYTES
IO_READ_RATE
Process read rate, in bytes per second.
DISK READ
IO_WRITE_RATE
Process write rate, in bytes per second.
DISK WRITE
IO_RATE
Process total I/O rate, in bytes per second.
DISK R/W
IO_PRIORITY
Process I/O priority.
IO
To add a parameter, select it and press F5 or Enter. To remove a parameter from Active Columns, select it and press F9.
When you finish managing columns, press F10 to save the changes and view the output.
3.5.5. Setting I/O Limits for Backup and Migration Operations¶
Backup and migration operations with containers and virtual machines can generate a high I/O load on the server, thus reducing the performance of other virtual environments or the server itself. You can avoid such situations by setting I/O limits for these operations.
To set an I/O limit, do the following:
In the
/etc/vz/vz.conf
global configuration file, locate the following section:# VZ Tools limits <...> # Uncomment next line to specify required disk IO bandwidth in Bps (10485760 - 10MBps) # VZ_TOOLS_IOLIMIT=10485760
Uncomment the
VZ_TOOLS_IOLIMIT
parameter, and set the I/O limit for backup and migration operations in bytes per second.Save the file.
When setting I/O limits, pay attention to the following:
VZ_TOOLS_IOLIMIT
is a global parameter that has effect on all virtual environments on the server.The
VZ_TOOLS_IOLIMIT
parameter controls the I/O load only for backup and migration operations. Restore operations are not limited.Simultaneous operations do not share the limit and are limited separately.
For migration, only the limit set on the source server applies, while the limit set on the destination server is ignored.
3.5.6. Improving Disk I/O Performance for Virtual Machines¶
In case of VM virtual disks mapped to QCOW2 disk images, the disk space is only allocated when it is actually needed by the VM and the cache size depends on how much space is allocated. In Virtuozzo Hybrid Server, you can significantly improve disk I/O performance by means of cache policies applied to VM disks based on their allocated space size.
The default cache policy is used for typical VM workloads and disk sizes up to 1TB. If a guest needs continuous access to an entire disk that is larger than 1TB, the default cache size will not be sufficient and will reduce disk I/O perfomance. In this case, you can increase cache size to speed up disk access as follows:
In the
/etc/libvirt/qemu.conf
configuration file, uncomment and set theqcow2_cache_policy
parameter todisk
.Restart the
libvirtd
daemon to apply the changes.# systemctl restart libvirtd
The disk
policy will provide a cache large enough to handle the entire disk workload.