2.13. Migrating Virtual Machines and Containers¶
To facilitate hardware upgrades and load balancing between multiple hosts, Virtuozzo Hybrid Server enables you to migrate virtual machines and containers either between Virtuozzo Hybrid Server 7 servers or from Virtuozzo 6 to Virtuozzo Hybrid Server 7 servers. Virtual machines and containers stored in a Virtuozzo Storage cluster are migrated between cluster nodes without data copying, which significantly reduces migration time.
Before migration, make sure the following requirements are met:
The destination server has enough hard disk space to store the resulting virtual machine or container and enough memory and CPU power to run it.
A stable network connection is provided between the source and destination servers.
A direct SSH connection on port 22 is allowed between the source and destination servers.
The source and destination servers must have exactly the same names for bridged networks used by virtual machines or containers.
Note the general restrictions that are applied to all types of migration:
Migration of virtual machines with snapshots is not supported (any existing snapshots must be deleted). For more information, see Managing Snapshots.
Migration of virtual machines with linked clones or linked clones of VMs is not supported. For more information, see Creating Linked Clones.
Migration of containers to virtual machines is not supported.
pmigrateutility is not supported.
Virtuozzo Hybrid Server allows you to perform two types of migration between Virtuozzo Hybrid Server servers:
Offline migration for stopped and suspended containers and virtual machines.
Online (live) migration for running containers and running and paused virtual machines. Containers and virtual machines may be located on Virtuozzo Storage or local storage.
Both types are described in the sections below.
2.13.1. Offline Migration of Virtual Machines and Containers¶
Offline migration implies copying all files of a virtual machine or container from one server to another over the network.
You can migrate VMs and containers both to and from a remote server. For example, to move a VM to a remote server, run this command on the local server:
# prlctl migrate MyVM remoteserver.com
To move a VM from a remote server, run this command the local server:
# prlctl migrate remoteserver.com/MyVM localhost
The root account is used to log in to the remote server by default, so you will be asked for the root password. You can also provide different credentials (and port) in the format [<user>[:<passwd>]@]<server>[:<port>]. Alternatively, you can set up RSA authentication and not use passwords at all (see Setting Up RSA Authentication Between Nodes).
If you want to place the migrated virtual environment in a custom directory on the destination server, specify the full path to that directory in the
--dst=<custom_path> option. The resulting path to the migrated virtual environment files will be
By default, once migration is complete:
the original virtual machine is removed from the source server,
.migratedsuffix is added to the names of the original container’s private area and configuration file on the source server.
--clone option to the
prlctl migrate command enables you to skip the two default actions above. That is, to keep the original VM and not to add the
.migrated suffix to container’s private area and configuration file. The clone will have a different UUID, MAC address, SID (for Windows-based VMs only; if the
--changesid option is specified), and offline management disabled.
For Windows 2019 VMs, use Microsoft’s Sysprep tool instead of the
Migration implies transferring large amounts of data between servers which can take considerable time. To reduce the amount of data to be transferred, Virtuozzo Hybrid Server has compression enabled by default. Compression consumes additional server resources and can be disabled if necessary with the
2.13.2. Live Migration of Virtual Machines and Containers¶
The process of migrating virtual machines and containers live is as follows:
Once you start the migration, Virtuozzo Hybrid Server checks whether the destination server meets all the migration requirements and the virtual machine or container can be migrated to it.
All virtual memory and disks of the virtual machine or container are migrated to the destination server.
The virtual machine or container on the source server is suspended.
The changed memory pages and virtual disk blocks, if any, are migrated to the destination server.
The virtual machine or container is resumed on the destination server.
The virtual machine or container continues running during steps 1 and 2 and is not available to the user during steps 3-5. But since the amount of memory pages and virtual disk blocks changed during step 2 is small, the downtime is almost imperceptible.
After migration, the relocated virtual machine or container may not be accessible over the network for several minutes due to network equipment reconfiguration (for example, as switches are updating their dynamic VLAN membership tables).
When performing live migration, take into account the following requirements and restrictions:
The system time on the source and destination servers should be synchronized, for example, by means of NTP (http://www.ntp.org). The reason is that certain processes running in virtual machines and containers may rely on system time being steady and might behave unpredictably when resumed on a destination server where time is different.
In Virtuozzo Hybrid Server 7, time synchronization via NTP is enabled by default using the
chronydservice. If you want to use
ntpd, stop and disable
The network must support data transfer rates of at least 1 Gbps.
The source and destination servers must belong to the same subnetwork.
The CPUs on the source and destination servers must be manufactured by the same vendor. The CPU features of the destination server must be the same or exceed those of the source server. If only some of the CPU features are common between the servers, you can create a CPU pool to guarantee live migration between them. For more information, see Managing CPU Pools.
Alternatively, you can mask individual CPU features with the
prlsrvctl set --cpu-features-maskcommand (refer to the
Both CPU pools and masking of individual CPU features only work on Intel Ivy Bridge and newer CPUs. They do not work on AMD processors.
Container live migration is only possible between nodes that have the
/proc/vz. For example:
# ls -l /proc/vz | grep cpuid_override -rw-r--r-- 1 root root 0 Feb 3 15:38 cpuid_override
Virtual machine and container disks can be located on local disks and Virtuozzo Storage.
Live migration is not supported for virtual machines and containers with open
prlctl entersessions and containers with IPSec connections.
Containers with NFS clients inside can be migrated only if the following conditions are met:
Block and character device files shared over NFS are not opened.
Remote file locking and over-mounted NFS file systems are not used simultaneously.
Migration of local file locks is supported only for NFS version 3 as it has native support of such locks.
Live migration is not supported for containers with enabled NFS server feature.
The command that launches online migration does not differ from that used for offline migration (see Offline Migration of Virtual Machines and Containers). For example, to move a running VM to a remote server, run this command on the local server:
# prlctl migrate MyVM remoteserver.com
For increased security during live migration, Virtuozzo Hybrid Server provides connection tunneling between the source and destination servers. Tunneling increases migration time. If you do not need a secure tunnel between servers, you can speed up VM live migration by disabling connection tunneling with the
--no-tunnel option. Tunnelless container migration is not supported. To use the option, configure the firewall of the destination server to allow incoming connections on any port on the corresponding network interface.
2.13.3. Migrating Virtual Machines and Containers from Virtuozzo 6 to Virtuozzo Hybrid Server 7¶
You can migrate older Virtuozzo 6 virtual machines (online) and containers (offline) to Virtuozzo Hybrid Server 7 servers. During migration, such VMs and containers will be converted in the Virtuozzo Hybrid Server 7 format. In particular, VM devices will be replaced by those supported in Virtuozzo Hybrid Server 7 (for a list of VM hardware supported in Virtuozzo Hybrid Server 7, see Virtual Machine Hardware).
Migration from Virtuozzo 6 to Virtuozzo Hybrid Server 7 implies VM and container downtime that depends on network bandwidth, virtual machine RAM size, and server load. To reduce downtime, it is recommended to at least perform migration when the server load is minimal.
Before migrating Virtuozzo 6 virtual machines, make sure these requirements are met in addition to those listed in Live Migration of Virtual Machines and Containers:
The hardware node must be running the latest version of Virtuozzo 6 (or at least 6.0.11-3466).
The VM must have a guest OS installed.
The VM must be running.
The VM must not have snapshots.
Windows VMs must have Virtuozzo guest tools and Linux VMs must have kernel headers installed.
For example, to migrate the virtual machine
MyVM from Virtuozzo 6 to Virtuozzo Hybrid Server 7, run the following command on the Virtuozzo 6 server:
# prlctl migrate MyVM <VZ7_host_IP_address_or_hostname>
The root account is used to log in to the remote server by default, so you will be asked for the root password. You can also provide different credentials (and port) in the format [<user>[:<passwd>]@]<server>[:<port>].
During migration, the virtual machine is copied to the destination server and converted to the Virtuozzo Hybrid Server 7 format. After a successful migration, the original Virtuozzo 6 virtual machine is unregistered and the
.migrated suffix is added to its directory name.
To avoid restarting migrated legacy VMs during conversion, such VMs have their
--on-crash parameter set to
paused (while the default value is
restart). After successful migration, check this parameter, e.g., with
prlctl list -i <VM_UUID|VM_name> | grep crash, and reset it if required with
prlctl set <VM_UUID|VM_name> --on-crash restart.
If conversion fails, the migrated VM is deleted from the destination server and you can try again. If the second attempt also fails, you need to enable a legacy VM debug mode on the destination Virtuozzo Hybrid Server 7 server (see Enabling Legacy VM Debug Mode), make another migration attempt, send the problem report, and contact the technical support team.
With the debug mode enabled, the migrated VM will not be deleted from the destination Virtuozzo Hybrid Server 7 server after conversion failure. It will remain stopped or running with disabled network to let the technical support team study the memory dump and find out the reason for failure.