10.7. Setting Up Docker and Kubernetes in Virtuozzo Containers¶
You can run Docker and Kubernetes inside Virtuozzo Hybrid Server system containers.
In the current version of Virtuozzo Hybrid Server, you can use system containers based on CentOS 7 as well as Ubuntu 18.04 and 20.04.
Note
Modules and third party add-ons that depend on operations prohibited in containers (loading of kernel modules, mounting of block devices, direct access to physical hardware) may not work in containers.
Essentially, you need to:
Enable and start the
vz-k8s-inside-ct
service that configures the hardware node for running Kubernetes and Docker swarm mode in system containers.Create a system container with a network connection and enough resources to run the Docker containers you need.
Install Docker inside the system container according to the official documentation. It is recommended to change the cgroup driver to
systemd
.Install Kubernetes inside the system container according to the official documentation.
Create more similar system containers and join them into a Kubernetes cluster or a Docker swarm.
These steps are demonstrated in more detail in the following tutorial:
Enable and start the
vz-k8s-inside-ct
service that configures the hardware node for Kubernetes and Docker swarm mode:# systemctl enable vz-k8s-inside-ct && systemctl start vz-k8s-inside-ct
Note
Restart running containers to apply these configuration changes to them.
Create a system container with a supported guest OS. For example:
# prlctl create dockerct1 --vmtype ct --ostemplate centos-7-x86_64
Provide the system container with enough resources. For example, add more RAM:
# prlctl set dockerct1 --memsize=2G Set the memsize parameter to 2048Mb.
Disable swap in the system container as required for Kubernetes. For example:
# prlctl set dockerct1 --swap=0 Set swappages 0
Set up network in the system container. For example, add a network adapter connected to a bridged network (by default) and enable DHCP for it:
# prlctl set dockerct1 --device-add net --dhcp yes Enable automatic reconfiguration for this network adapter. Creating net0 (+) dev='' ifname='eth0' network='Bridged' mac=<...> card=virtio dhcp='yes' Created net0 (+) dev='veth72d71c9b' ifname='eth0' network='Bridged' mac=<...> dhcp='yes'
Set a unique hostname for the system container. For example:
# prlctl set dockerct1 --hostname dockerct1.example.local
Note
Make sure to register the hostname with the DNS server.
Set the root password for the system container to be able to log in via SSH:
# prlctl set dockerct1 --userpasswd root:<passwd>
Where
<passwd>
is a strong password.Start and log in to the system container:
# prlctl start dockerct1 # ssh <dockerct1_IP>
Where
<dockerct1_IP>
is the system container’s IP address. If you previously enabled DHCP, you may need to find out the IP address. For example:# prlctl exec dockerct1 ip -4 a | grep inet <...> inet 10.194.110.194/16 brd 10.194.255.255 scope global dynamic eth0
Perform the next steps inside the system container.
Open the ports required by Kubernetes and the Docker swarm:
For Kubernetes control-plane:
# firewall-cmd --permanent --add-port=6443/tcp;\ firewall-cmd --permanent --add-port=2379-2380/tcp;\ firewall-cmd --permanent --add-port=10250/tcp;\ firewall-cmd --permanent --add-port=10251/tcp;\ firewall-cmd --permanent --add-port=10252/tcp;\ firewall-cmd --reload
For the Docker swarm:
# firewall-cmd --permanent --add-port=2377/tcp;\ firewall-cmd --permanent --add-port=7946/tcp;\ firewall-cmd --permanent --add-port=7946/udp;\ firewall-cmd --permanent --add-port=4789/udp;\ firewall-cmd --reload
Install Docker as laid out in the official guide. Before you start the Docker service, however, change the cgroup driver to
systemd
:# mkdir /etc/docker # cat > /etc/docker/daemon.json <<EOF { "exec-opts": [ "native.cgroupdriver=systemd" ], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "storage-driver": "overlay2", "storage-opts": [ "overlay2.override_kernel_check=true" ] } EOF
Enable and start the Docker service:
# systemctl enable docker && systemctl daemon-reload && systemctl start docker
Note
If the Docker service fails to start with the non-mandatory option
storage-opts
, remove this option. The final configuration is as follows:# cat > /etc/docker/daemon.json <<EOF { "exec-opts": [ "native.cgroupdriver=systemd" ], "log-driver": "json-file", "log-opts": { "max-size": "100m" }, "storage-driver": "overlay2" } EOF
Verify that Docker Engine works:
# docker run hello-world Unable to find image 'hello-world:latest' locally latest: Pulling from library/hello-world 0e03bdcc26d7: Pull complete Digest: sha256:4cf9c47f86df71d48364001ede3a4fcd85ae80ce02ebad74156906caff5378bc Status: Downloaded newer image for hello-world:latest Hello from Docker! This message shows that your installation appears to be working correctly. <...>
Install
kubelet
,kubeadm
, andkubectl
packages according to the official documentation.Initialize the Kubernetes control-plane node. For example:
# kubeadm init <...> Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 10.194.110.194:6443 --token qrdi2v.t4rr9yyylpgpnkcw \ --discovery-token-ca-cert-hash sha256:a7d0e65<...>
As you are the root user, run
# export KUBECONFIG=/etc/kubernetes/admin.conf
Deploy a pod network. For example:
# kubectl apply -f "https://cloud.weave.works/k8s/net?\ k8s-version=$(kubectl version | base64 | tr -d '\n')" serviceaccount/weave-net created clusterrole.rbac.authorization.k8s.io/weave-net created clusterrolebinding.rbac.authorization.k8s.io/weave-net created role.rbac.authorization.k8s.io/weave-net created rolebinding.rbac.authorization.k8s.io/weave-net created daemonset.apps/weave-net created
Check that Kubernetes has been set up correctly. For example:
# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-f9fd979d6-29k27 1/1 Running 0 5m kube-system coredns-f9fd979d6-f6xd5 1/1 Running 0 5m kube-system etcd-dockerct1.example.local 1/1 Running 0 5m kube-system kube-apiserver-dockerct1.example.local 1/1 Running 0 5m kube-system kube-controller-manager-dockerct1.example.local 1/1 Running 0 5m kube-system kube-proxy-559tc 1/1 Running 0 5m kube-system kube-scheduler-dockerct1.example.local 1/1 Running 0 5m kube-system weave-net-jzm8p 2/2 Running 0 5m
Create a Docker swarm using the system container’s IP address. For example:
# docker swarm init --advertise-addr 10.194.110.194 Swarm initialized: current node (n1slc7on484f7ak4d3vaz6tk4) is now a manager. To add a worker to this swarm, run the following command: docker swarm join --token SWMTKN-1-<...> 10.194.110.194:2377 To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
Create more (e.g., two more) system containers with the same configuration as explained in steps 2 to 8. They will run Kubernetes and Docker swarm workers.
Perform the next steps in each of the new system containers.
Open the ports required by Kubernetes and the Docker swarm:
For the Kubernetes worker:
# firewall-cmd --permanent --add-port=10250/tcp;\ firewall-cmd --permanent --add-port=30000-32767/tcp;\ firewall-cmd --reload
For the Docker swarm:
# firewall-cmd --permanent --add-port=2377/tcp;\ firewall-cmd --permanent --add-port=7946/tcp;\ firewall-cmd --permanent --add-port=7946/udp;\ firewall-cmd --permanent --add-port=4789/udp;\ firewall-cmd --reload
Install Docker and Kubernetes as explained in steps 10 to 13.
Add workers to the Kubernetes cluster with the command reported by
kubeadm init
. For example:# kubeadm join 10.194.110.194:6443 --token qrdi2v.t4rr9yyylpgpnkcw \ --discovery-token-ca-cert-hash sha256:a7d0e65<...>
Add workers to the Docker swarm with the command reported by
docker swarm init
. For example:# docker swarm join --token SWMTKN-1-<...> 10.194.110.194:2377 This node joined a swarm as a worker.
Log in to the system container running the control plane and swarm manager (
dockerct1
in this tutorial) and check that both the Kubernetes cluster and Docker swarm work. For example:# kubectl get nodes NAME STATUS ROLES AGE VERSION dockerct1.example.local Ready master 10m v1.19.2 dockerct2.example.local Ready <none> 5m v1.19.2 dockerct3.example.local Ready <none> 5m v1.19.2
# docker info <...> Swarm: active NodeID: n1slc7on484f7ak4d3vaz6tk4 Is Manager: true ClusterID: lyz9u13zoswtm03jkswqcdzaj Managers: 1 Nodes: 3 Default Address Pool: 10.0.0.0/8 SubnetSize: 24 Data Path Port: 4789 Orchestration: Task History Retention Limit: 5 Raft: Snapshot Interval: 10000 Number of Old Snapshots to Retain: 0 Heartbeat Tick: 1 Election Tick: 10 Dispatcher: Heartbeat Period: 5 seconds CA Configuration: Expiry Duration: 3 months Force Rotate: 0 Autolock Managers: false Root Rotation In Progress: false Node Address: 10.194.110.194 Manager Addresses: 10.194.110.194:2377 <...>
After completing this tutorial, you have three or more Virtuozzo Hybrid Server 7 system containers running Docker and Kubernetes. You have also created a Kubernetes cluster and a Docker swarm.