10.27. Managing Windows Containers¶
Important
This feature is a technical preview. It is not ready for production.
You can use Virtuozzo Hybrid Server to create and manage Windows containers on Microsoft Windows nodes.
10.27.1. Setting Up the Environment¶
You will need the following:
A node running Virtuozzo Hybrid Server 7.5 or newer.
Three or more Microsoft Windows nodes in a Storage Spaces Direct cluster. The nodes must be running Microsoft Windows Server 2019 v1809 or newer. It is also implied that the nodes meet the Storage Spaces Direct hardware requirements.
A network interconnecting the Virtuozzo Hybrid Server node and Windows nodes. All Microsoft Windows nodes will need Internet access for Docker installation.
10.27.1.1. Configuring the Virtuozzo Hybrid Server Node¶
Install the required packages:
# yum groupinstall "Windows Containers" # yum install virt-install
Create a temporary directory, e.g,
/home/tmp/
, and generate certificates in it.The CA certificate:
# /usr/share/winrm-docker-deploy/create-certs.sh -m ca \ -t /home/tmp/ -pw <cert_passwd>
Where
<cert_passwd>
is a password for the certificate.The client certificate:
# /usr/share/winrm-docker-deploy/create-certs.sh -m client \ -t /home/tmp/ -us <windows_admin> -pw <cert_passwd>
Where
<windows_admin>
is the username of an account that is in the Administrators group on every cluster node.A certificate for every node in the Storage Spaces Direct cluster:
# /usr/share/winrm-docker-deploy/create-certs.sh -m server \ -h <hostname> -hip <hostip> -t /home/tmp/ -pw <cert_passwd>
Where
<hostname>
and<hostip>
are the host name and IP address of a cluster node, respectively. Run this command for each cluster node.
For example, if you have three nodes in the Storage Spaces Direct cluster: win1.example.com, win2.example.com, and win3.example.com, you will get these certificates as a result:
# ls -1 /home/tmp/ cakey.pem ca.pem ca.srl cert.pem key.pem win1.example.com-https.pfx win1.example.com-server-cert.pem win1.example.com-server-key.pem win2.example.com-https.pfx win2.example.com-server-cert.pem win2.example.com-server-key.pem win3.example.com-https.pfx win3.example.com-server-cert.pem win3.example.com-server-key.pem
Note
Non-root users may require additional permissions to read these files.
Copy the CA certificate as well as the client public and private certificates to
/etc/pki/libvirt/docker/
:# cp /home/tmp/{ca,cert,key}.pem /etc/pki/libvirt/docker/
10.27.1.2. Configuring Microsoft Windows Nodes¶
Perform the following steps on each Windows Server node in the Storage Spaces Direct cluster:
Install the Containers and Windows Server Backup features. For example, from a PowerShell session run as administrator:
PS C:\> Install-WindowsFeature -Name Containers, Windows-Server-Backup
Reboot the node.
Download the deployment bundle and unpack it to a temporary directory, e.g.,
C:\ProgramData\Virtuozzo\docker-bundle\
. For example:PS C:\> (New-Object System.Net.WebClient).DownloadFile( ` "https://docs.virtuozzo.com/files/winct/docker-deploy-latest.zip", ` "C:\docker-deploy-latest.zip")
PS C:\> Expand-Archive -LiteralPath C:\docker-deploy-latest.zip ` -DestinationPath C:\ProgramData\Virtuozzo\docker-bundle\
You will get the following files as a result:
PS C:\> ls C:\ProgramData\Virtuozzo\docker-bundle\ Directory: C:\ProgramData\Virtuozzo\docker-bundle\ Mode LastWriteTime Length Name ---- ------------- ------ ---- d----- 11/26/2020 8:03 AM bindata d----- 11/26/2020 8:03 AM certs d----- 11/26/2020 8:03 AM vz-windows-backup -a---- 11/25/2020 3:25 PM 38131 auto-deploy.ps1 -a---- 11/25/2020 3:25 PM 140 bundle-version.txt -a---- 11/25/2020 3:25 PM 27905 winct-uninstall.ps1
Copy the CA and node’s certificates from the Virtuozzo Hybrid Server node to
C:\ProgramData\Virtuozzo\docker-bundle\certs
. Remove the hostname from file names. For example:PS C:\> scp root@10.94.106.235:/etc/pki/libvirt/docker/ca.pem ` C:\ProgramData\Virtuozzo\docker-bundle\certs\ root@10.94.106.235's password: ca.pem 100% 1781 1.7KB/s 00:00
PS C:\> scp root@10.94.106.235:/home/tmp/$(` [System.Net.Dns]::GetHostByName($env:COMPUTERNAME).hostname)* ` C:\ProgramData\Virtuozzo\docker-bundle\certs\ root@10.94.106.235's password: win1.example.com-https.pfx 100% 4021 3.9KB/s 00:00 win1.example.com-server-cert.pem 100% 1777 1.7KB/s 00:00 win1.example.com-server-key.pem 100% 3243 3.2KB/s 00:00
PS C:\> ls C:\ProgramData\Virtuozzo\docker-bundle\certs\ | Rename-Item -NewName ` { $_.Name -replace $([System.Net.Dns]::GetHostByName($env:COMPUTERNAME).hostname+"-"),'' }
You will get the following files as a result:
PS C:\> ls C:\ProgramData\Virtuozzo\docker-bundle\certs\ Directory: C:\ProgramData\Virtuozzo\docker-bundle\certs\ Mode LastWriteTime Length Name ---- ------------- ------ ---- -ar--- 11/4/2020 11:42 AM 1781 ca.pem -a---- 11/4/2020 11:41 AM 4021 https.pfx -ar--- 11/4/2020 11:41 AM 1777 server-cert.pem -ar--- 11/4/2020 11:41 AM 3243 server-key.pem
Run the deployment script:
PS C:\> cd C:\ProgramData\Virtuozzo\docker-bundle\ PS C:\> .\auto-deploy.ps1 -username:<windows_admin> ` -password:<windows_admin_passwd> -pwforpfx:<cert_passwd>
Where
<windows_admin>
and<windows_admin_passwd>
are the username and password of an account that is in the Administrators group on every cluster node and<cert_passwd>
is the certificate password. For simplicity, use the same administrator account that you generated the certificates with.The deployment script will download and install Docker and required components, including a base Nano Server 1809 image. It will also replace the original
dockerd.exe
with a custom one made by Virtuozzo. If Docker is already installed, the script will update it and replace the executable. The script will also add firewall rules and import the certificates. The log will be saved toC:\ProgramData\Virtuozzo\winctfiles\auto-deploy.log
.Note
If you only need to import or update the certificates, use the deployment script with the
-onlycerts
option. For more options, run.\auto-deploy.ps1 -help
.
After the script finishes, reboot the node and check that the Docker client and server are running. For example:
PS C:\> docker version
Client: Docker Engine - Enterprise
Version: 19.03.12
API version: 1.40
Go version: go1.13.13
Git commit: 4306744
Built: 08/05/2020 19:27:53
OS/Arch: windows/amd64
Experimental: false
Server:
Engine:
Version: 19.03.13
API version: 1.40 (minimum version 1.24)
Go version: go1.13.15
Git commit: cb7af2e
Built: Tue Oct 13 12:15:38 2020
OS/Arch: windows/amd64
Experimental: false
If an error is shown for the server, try starting it manually:
PS C:\> Start-Service docker
10.27.2. Creating Windows Containers¶
After setting up both the Virtuozzo Hybrid Server and Microsoft Windows Server nodes, you can create Windows containers using virt-install
. Container base images need to be present on Windows Server nodes. The deployment script only installs the Nano Server 1809 image. Any other base images need to be pulled from the Docker Hub first.
You can create a Windows container from the Virtuozzo Hybrid Server node as follows:
# virt-install --connect docker://<hostname> --name <ct_name> --memory 1024 \
--boot init=* --os-variant none --console none --disk bus=virtio,path=<ct_image> \
--network type=network,mac=<ct_MAC>,source=<net_name> \
--filesystem <guest_dir>,<host_dir> --noreboot --noautoconsole
Where:
<hostname>
is the hostname of the Windows Server node to create the container on.<ct_name>
is a unique container name.<ct_image>
is a base image to create the container from. For example,mcr.microsoft.com/windows/nanoserver:1809
.<ct_MAC>
is a unique container MAC address.<net_name>
is a name of a Docker network to connect the container to. Usedefault
for the default NAT network.<guest_dir>
and<host_dir>
are the container guest and host directories located on a virtual disk in the Storage Spaces Direct cluster. Create these directories if they do not exist.
For example:
# virt-install --connect docker://win1.example.com --name winct1 --memory 1024 \
--boot init=* --os-variant none --console none \
--disk bus=virtio,path=mcr.microsoft.com/windows/nanoserver:1809 \
--network type=network,mac=00:01:02:03:04:05,source=default \
--filesystem C:/ClusterStorage/disk1/guest_dir/,C:/ClusterStorage/disk1/host_dir/ \
--noreboot --noautoconsole
Starting install...
Domain creation completed.
You can restart your domain by running:
virsh --connect docker://win1.example.com start winct1
This command will create a Windows container based on a Nano Server image.
As more examples for this technical preview, you can also create Windows containers with DotNetNuke and WordPress from base images published by Virtuozzo at Docker Hub. These base images include the required database management systems, so you can skip to setting up DotNetNuke and WordPress once you run the container. Do the following:
Pull the images from Docker Hub to Windows Server nodes:
PS C:\> docker pull virtuozzo/dnn:techpreview
Or
PS C:\> docker pull virtuozzo/wordpress:techpreview
Create the Windows container from the Virtuozzo Hybrid Server node. For example:
# virt-install --connect docker://win1.example.com --name winct1dnn --memory 2048 \ --boot init=* --os-variant none --console none \ --disk bus=virtio,path=virtuozzo/dnn:techpreview \ --network type=network,mac=00:01:02:03:04:06,source=default \ --filesystem C:/ClusterStorage/disk1/guest_dir/,C:/ClusterStorage/disk1/host_dir/ \ --noreboot --noautoconsole # virsh --connect docker://win1.example.com start winct1dnn
Provide A DotNetNuke container with at least 2GB of memory for the setup to complete.
Or
# virt-install --connect docker://win1.example.com --name winct1wp --memory 1024 \ --boot init=* --os-variant none --console none \ --disk bus=virtio,path=virtuozzo/wordpress:techpreview \ --network type=network,mac=00:01:02:03:04:07,source=default \ --filesystem C:/ClusterStorage/disk1/guest_dir/,C:/ClusterStorage/disk1/host_dir/ \ --noreboot --noautoconsole # virsh --connect docker://win1.example.com start winct1wp
Find out the container’s IP address. For example:
# virsh --connect docker://win1.example.com dumpxml winct1dnn | grep "ip address" <ip address='192.168.175.204' family='ipv4' prefix='24'/>
Or
# virsh --connect docker://win1.example.com dumpxml winct1wp | grep "ip address" <ip address='192.168.175.114' family='ipv4' prefix='24'/>
As the Windows containers use a NAT network by default, they are only accessible from the Windows Server node the container is running on. From that node, visit the container’s IP address (and optionally port) in a web browser to proceed with setup.
To make Windows containers publicly accessible, let them use a virtual switch bound to a node’s physical network adapter connected to a public network. To create a virtual switch, do the following:
Open the Hyper-V Manager. E.g., in Server Manager, click Tools > Hyper-V Manager or run
virtmgmt.msc
from the Command Prompt or PowerShell.In the Hyper-V Manager, click Virtual Switch Manager in the Actions menu.
In the Virtual Switch Manager, choose New virtual network switch > External. Click Create Virtual Switch.
In Virtual Switch Properties, enter a name for the switch, e.g., vSwitch1, and select a public physical network adapter in External network. Click OK.
Restart Docker:
PS C:\> Restart-Service docker
Check that the switch is listed in Docker networks. For example:
PS C:\> docker network ls NETWORK ID NAME DRIVER SCOPE ad2c3ef4de3e nat nat local ea6c5aa76c42 none null local 7ae7bd5c638e vSwitch1 transparent local
Now you can specify the public network, e.g., --network ... source=vSwitch1
instead of --network ... source=default
, when creating Windows containers with virt-install
.
10.27.3. Starting, Listing, and Stopping Windows Containers¶
After creating a Windows container, start it as indicated by virt-install
. For example:
# virsh --connect docker://win1.example.com start winct1
Domain winct1 started
Using virsh
, you can also list and shut down Windows containers on the node. For example:
# virsh --connect docker://win1.example.com list --all
Id Name State
-----------------------------
39370 winct1 running
# virsh --connect docker://win1.example.com shutdown winct1
10.27.4. Running Commands in Windows Containers¶
To attach to a Windows container in this technical preview, use the docker attach
command on the Windows Server node the container is running on.
To run a command inside a Windows container, use the docker exec
command as follows:
PS C:\> docker exec <ct> <shell> /c "<command>"
Where:
<ct>
is the container name or ID.<shell>
is the name of the shell bundled with the container image, e.g.,cmd
is Command Prompt andpwsh
is PowerShell.<command>
is the command to execute in the shell.
For example:
PS C:\> docker exec winct1 cmd /c "echo Test > C:\ClusterStorage\disk1\host_dir\test.txt"
PS C:\> docker exec winct1 cmd /c "type C:\ClusterStorage\disk1\host_dir\test.txt"
Test
PS C:\> docker exec winct1 cmd /c "dir C:\ClusterStorage\disk1\host_dir\"
Volume in drive C has no label.
Volume Serial Number is 34FA-4299
Directory of C:\ClusterStorage\disk1\host_dir
11/05/2020 03:04 AM <DIR> .
11/05/2020 03:04 AM <DIR> ..
11/05/2020 03:01 AM 9 test.txt
1 File(s) 9 bytes
2 Dir(s) 544,967,794,688 bytes free
10.27.5. Configuring Windows Container Parameters¶
In this technical preview, redefining container configuration (editing its XML scheme) is not supported. You can, however, update container resource limits as follows:
Change available memory using
virsh setmem
. For example:# virsh -c docker://win1.example.com setmem winct1 512M # virsh -c docker://win1.example.com dumpxml winct1 | grep -i memory <memory unit='KiB'>524288</memory> <currentMemory unit='KiB'>524288</currentMemory>
Edit the CPU shares or quotas for the container using
virsh schedinfo
. For example:# virsh -c docker://win1.example.com schedinfo winct1 cpu_shares=1000 Scheduler : posix cpu_shares : 1000 vcpu_period : 0 vcpu_quota : 0
Or
# virsh -c docker://win1.example.com schedinfo winct1 vcpu_period=4000 vcpu_quota=2000 Scheduler : posix cpu_shares : 0 vcpu_period : 4000 vcpu_quota : 2000
You cannot set both
cpu_shares
andvcpu_quota
at the same time.Set
cpu_shares
orvcpu_quota
or to -1 to reset it:# virsh -c docker://win1.example.com schedinfo winct1 cpu_shares=-1
Or
# virsh -c docker://win1.example.com schedinfo winct1 vcpu_quota=-1
Throttle container disk with
virsh blkdeviotune
by changing the total throughput limit, in bytes per second, as well as the total I/O operations limit per second. For example:# virsh -c docker://win1.example.com blkdeviotune winct1 vda \ --total-bytes-sec=104857600 --total-iops-sec=1000
Use
virsh domblklist
to find out container’s device names.
10.27.6. Migrating Windows Containers¶
Take note of the following:
The source container must be stopped.
The name and UUID of the migrated container must be unique on the destination node.
The migrated container keeps the source’s UUID.
The MAC address of the migrated container must be unique within the destination node’s network. For example, if all nodes belong to the same network, MAC addresses must be unique across all nodes.
The data on container’s drive C: will be lost after migration. The data on persistent volumes will remain.
To migrate a Windows container, use the migrate
subcommand of virsh
. For example:
Migrate a container and delete the source:
# virsh --connect docker://node3.winctpreview.com migrate winct1 --desturi \ docker://node1.winctpreview.com --offline --persistent --undefinesource
Migrate a container under a new name and delete the source:
# virsh --connect docker://node3.winctpreview.com migrate winct1 --desturi \ docker://node1.winctpreview.com --offline --persistent --undefinesource \ --dname winct1_migrated
Migrate a container under a new name and keep the source:
# virsh --connect docker://node3.winctpreview.com migrate winct1 --desturi \ docker://node1.winctpreview.com --dname winct1_migrated --offline --persistent
Note
In this case, the container can only be migrated once. On the second attempt, the UUIDs of the source container and the one migrated earlier will be the same, which will cause a conflict. You can, however, create a container with a new UUID from the source’s configuration file and migrate it.
10.27.7. Managing Windows Container Backups¶
It is recommended to use the Windows Server Backup feature for container backups. You can use the provided PowerShell scripts to partially automate backup-related tasks.
Run the script on each node to find the containers with persistent CSVs and add those volumes to the backup schedule:
PS> Config-Docker-Backup.ps1 -Times '<1st> <2nd> <3rd>'
The script will configure the Windows Server Backup feature to create incremental backups daily at <1st>
, <2nd>
, and <3rd>
. If you omit the -Times
parameter, backups will be created at 9:00, 12:00, and 15:00.
Container backups will be listed in the Windows Server Backup snap-in. To open it, run wbadmin.msc
.
To restore a container backup, run
PS> Restore-Docker.ps1 <ct_name>
Where <ct_name>
is container name.
Restoring will overwrite the backed up container volume(s).
10.27.8. Troubleshooting Windows Containers¶
In case of issues with Windows containers, you can create a problem report and send it to the technical support team:
usage: vz-problem-report [-h] [-v] [--proxy PROXY] [--name NAME]
[--email EMAIL] [--description DESC]
[--reportfile FILE] [--version]
address
Where
address
is the hostname or IP address of the Microsoft Windows Server host to collect the report fromname
andemail
are your name and email address, respectivelydesc
is the description of the problemreportfile
is the name of the report file to save locally
If the node is not connected to the Internet, you can create a local report file with the --reportfile
option and manually email it to the support team.
10.27.9. Uninstalling the Windows Containers Feature¶
To remove the ability to create and manage Windows Containers on Microsoft Windows Server nodes from a Virtuozzo Hybrid Server node, run the uninstallation script:
PS C:\> C:\Program Files\Virtuozzo\winctfiles\winct-uninstall.ps1
The script will attempt to undo changes made by the deployment script:
Uninstall Docker if it was not installed before the deployment. If Docker was already installed, leave it with the custom
dockerd.exe
but restore its service’s original startup type (e.g., Automatic or Manual). Stop the service if the original startup type is Manual.Uninstall DockerMsftProvider.
Delete joblib, Prometheus exporter, and backup scripts.
Remove the path to the bundle from
PATH
.Restore the original startup type of the WinRM service. Stop the service if the original startup type is Manual.
Delete the firewall rules for Docker and WinRM. If rules existed and were changed, restore them.
Delete the imported certificate files from ProgramDatadockercerts.d. Delete the certificates from the store. Kill the HTTP Listener in WinRM and remove authentication by the imported certificate.