Prev Next

10.7. Setting Up Docker and Kubernetes in Virtuozzo Containers¶

You can run Docker and Kubernetes inside Virtuozzo Hybrid Server system containers.

In the current version of Virtuozzo Hybrid Server, you can use system containers based on CentOS 7 as well as Ubuntu 18.04 and 20.04.

Note

Modules and third party add-ons that depend on operations prohibited in containers (loading of kernel modules, mounting of block devices, direct access to physical hardware) may not work in containers.

Essentially, you need to:

Enable and start the vz-k8s-inside-ct service that configures the hardware node for running Kubernetes and Docker swarm mode in system containers.
Create a system container with a network connection and enough resources to run the Docker containers you need.
Install Docker inside the system container according to the official documentation. It is recommended to change the cgroup driver to systemd.
Install Kubernetes inside the system container according to the official documentation.
Create more similar system containers and join them into a Kubernetes cluster or a Docker swarm.

These steps are demonstrated in more detail in the following tutorial:

Enable and start the vz-k8s-inside-ct service that configures the hardware node for Kubernetes and Docker swarm mode:
```
# systemctl enable vz-k8s-inside-ct && systemctl start vz-k8s-inside-ct
```
Note

Restart running containers to apply these configuration changes to them.

Create a system container with a supported guest OS. For example:

# prlctl create dockerct1 --vmtype ct --ostemplate centos-7-x86_64

Provide the system container with enough resources. For example, add more RAM:
```
# prlctl set dockerct1 --memsize=2G
Set the memsize parameter to 2048Mb.
```
Disable swap in the system container as required for Kubernetes. For example:
```
# prlctl set dockerct1 --swap=0
Set swappages 0
```

Set up network in the system container. For example, add a network adapter connected to a bridged network (by default) and enable DHCP for it:

# prlctl set dockerct1 --device-add net --dhcp yes
Enable automatic reconfiguration for this network adapter.
Creating net0 (+) dev='' ifname='eth0' network='Bridged' mac=<...> card=virtio dhcp='yes'
Created net0 (+) dev='veth72d71c9b' ifname='eth0' network='Bridged' mac=<...> dhcp='yes'

Set a unique hostname for the system container. For example:
```
# prlctl set dockerct1 --hostname dockerct1.example.local
```
Note

Make sure to register the hostname with the DNS server.
Set the root password for the system container to be able to log in via SSH:
```
# prlctl set dockerct1 --userpasswd root:<passwd>
```
Where <passwd> is a strong password.
Start and log in to the system container:
```
# prlctl start dockerct1
# ssh <dockerct1_IP>
```
Where <dockerct1_IP> is the system container’s IP address. If you previously enabled DHCP, you may need to find out the IP address. For example:
```
# prlctl exec dockerct1 ip -4 a | grep inet
<...>
inet 10.194.110.194/16 brd 10.194.255.255 scope global dynamic eth0
```
Perform the next steps inside the system container.

Open the ports required by Kubernetes and the Docker swarm:

For Kubernetes control-plane:

# firewall-cmd --permanent --add-port=6443/tcp;\
firewall-cmd --permanent --add-port=2379-2380/tcp;\
firewall-cmd --permanent --add-port=10250/tcp;\
firewall-cmd --permanent --add-port=10251/tcp;\
firewall-cmd --permanent --add-port=10252/tcp;\
firewall-cmd --reload

For the Docker swarm:

# firewall-cmd --permanent --add-port=2377/tcp;\
firewall-cmd --permanent --add-port=7946/tcp;\
firewall-cmd --permanent --add-port=7946/udp;\
firewall-cmd --permanent --add-port=4789/udp;\
firewall-cmd --reload

Install Docker as laid out in the official guide. Before you start the Docker service, however, change the cgroup driver to systemd:

# mkdir /etc/docker
# cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": [
    "native.cgroupdriver=systemd"
  ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}
EOF

Enable and start the Docker service:

# systemctl enable docker && systemctl daemon-reload && systemctl start docker

Note

If the Docker service fails to start with the non-mandatory option storage-opts, remove this option. The final configuration is as follows:

# cat > /etc/docker/daemon.json <<EOF
{
  "exec-opts": [
    "native.cgroupdriver=systemd"
  ],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2"
}
EOF

Verify that Docker Engine works:

# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete
Digest: sha256:4cf9c47f86df71d48364001ede3a4fcd85ae80ce02ebad74156906caff5378bc
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.
<...>

Install kubelet, kubeadm, and kubectl packages according to the official documentation.

Initialize the Kubernetes control-plane node. For example:

# kubeadm init
<...>
Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.194.110.194:6443 --token qrdi2v.t4rr9yyylpgpnkcw \
    --discovery-token-ca-cert-hash sha256:a7d0e65<...>

As you are the root user, run

# export KUBECONFIG=/etc/kubernetes/admin.conf

Deploy a pod network. For example:

# kubectl apply -f "https://cloud.weave.works/k8s/net?\
k8s-version=$(kubectl version | base64 | tr -d '\n')"
serviceaccount/weave-net created
clusterrole.rbac.authorization.k8s.io/weave-net created
clusterrolebinding.rbac.authorization.k8s.io/weave-net created
role.rbac.authorization.k8s.io/weave-net created
rolebinding.rbac.authorization.k8s.io/weave-net created
daemonset.apps/weave-net created

Check that Kubernetes has been set up correctly. For example:

# kubectl get pods --all-namespaces
NAMESPACE     NAME                                             READY  STATUS   RESTARTS  AGE
kube-system   coredns-f9fd979d6-29k27                          1/1    Running  0         5m
kube-system   coredns-f9fd979d6-f6xd5                          1/1    Running  0         5m
kube-system   etcd-dockerct1.example.local                     1/1    Running  0         5m
kube-system   kube-apiserver-dockerct1.example.local           1/1    Running  0         5m
kube-system   kube-controller-manager-dockerct1.example.local  1/1    Running  0         5m
kube-system   kube-proxy-559tc                                 1/1    Running  0         5m
kube-system   kube-scheduler-dockerct1.example.local           1/1    Running  0         5m
kube-system   weave-net-jzm8p                                  2/2    Running  0         5m

Create a Docker swarm using the system container’s IP address. For example:

# docker swarm init --advertise-addr 10.194.110.194
Swarm initialized: current node (n1slc7on484f7ak4d3vaz6tk4) is now a manager.
To add a worker to this swarm, run the following command:
    docker swarm join --token SWMTKN-1-<...> 10.194.110.194:2377
To add a manager to this swarm, run 'docker swarm join-token manager'
and follow the instructions.

Create more (e.g., two more) system containers with the same configuration as explained in steps 2 to 8. They will run Kubernetes and Docker swarm workers.

Perform the next steps in each of the new system containers.

Open the ports required by Kubernetes and the Docker swarm:

For the Kubernetes worker:

# firewall-cmd --permanent --add-port=10250/tcp;\
firewall-cmd --permanent --add-port=30000-32767/tcp;\
firewall-cmd --reload

For the Docker swarm:

# firewall-cmd --permanent --add-port=2377/tcp;\
firewall-cmd --permanent --add-port=7946/tcp;\
firewall-cmd --permanent --add-port=7946/udp;\
firewall-cmd --permanent --add-port=4789/udp;\
firewall-cmd --reload

Install Docker and Kubernetes as explained in steps 10 to 13.

Add workers to the Kubernetes cluster with the command reported by kubeadm init. For example:

# kubeadm join 10.194.110.194:6443 --token qrdi2v.t4rr9yyylpgpnkcw \
      --discovery-token-ca-cert-hash sha256:a7d0e65<...>

Add workers to the Docker swarm with the command reported by docker swarm init. For example:

# docker swarm join --token SWMTKN-1-<...> 10.194.110.194:2377
This node joined a swarm as a worker.

Log in to the system container running the control plane and swarm manager (dockerct1 in this tutorial) and check that both the Kubernetes cluster and Docker swarm work. For example:

# kubectl get nodes
NAME                      STATUS   ROLES    AGE   VERSION
dockerct1.example.local   Ready    master   10m   v1.19.2
dockerct2.example.local   Ready    <none>   5m    v1.19.2
dockerct3.example.local   Ready    <none>   5m    v1.19.2

# docker info
<...>
Swarm: active
 NodeID: n1slc7on484f7ak4d3vaz6tk4
 Is Manager: true
 ClusterID: lyz9u13zoswtm03jkswqcdzaj
 Managers: 1
 Nodes: 3
 Default Address Pool: 10.0.0.0/8
 SubnetSize: 24
 Data Path Port: 4789
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 10.194.110.194
 Manager Addresses:
  10.194.110.194:2377
<...>

After completing this tutorial, you have three or more Virtuozzo Hybrid Server 7 system containers running Docker and Kubernetes. You have also created a Kubernetes cluster and a Docker swarm.

Version 7.5 — Apr 22, 2024

Edit Print

Prev Next