10.1. Configuring Automatic Memory Management Policies¶

10.1.1. Restarting VCMMD¶

In certain situations, you may need to restart the VCMMD daemon. For example:

• When you activate a license after installing Virtuozzo Hybrid Server or update an expired license.

• To permanently apply custom limits and guarantees for services residing in memory slices. See Managing Host Services with VCMMD and Managing Virtuozzo Storage Services with VCMMD.

• If you deploy Virtuozzo Storage on top of Virtuozzo Hybrid Server and start its services manually without rebooting the server. See Managing Virtuozzo Storage Services with VCMMD.

• To apply changes after tweaking the VCMMD configuration file. See Managing Virtuozzo Storage Services with VCMMD.

If virtual environments are running on the server, do the following:

1. Stop all virtual environments on the server or temporarily live-migrate them to another server.

2. Restart vcmmd:

   # systemctl restart vcmmd
   

   The default performance policy will be enabled.

3. Start the virtual environments or live-migrate them back to the server, depending on what you did in step 1.

If no virtual environments are running on the server, just restarting the service is enough.

10.1.2. Optimizing Virtual Machine Memory Usage with Kernel Same-Page Merging¶

To optimize memory usage by virtual machines, Virtuozzo Hybrid Server uses a Linux feature called kernel same-page merging (KSM). The KSM daemon periodically scans memory for pages with identical content and merges those into a single page. That page is marked as copy-on-write (COW). When a virtual machine needs to change that page’s contents, the kernel creates a copy of the page for that VM and changes it.

KSM enables the server to:

• Avoid swapping

• Run more virtual machines

• Overcommit virtual machine memory

• Speed up RAM and hence certain applications and guest operating systems

In Virtuozzo Hybrid Server 7, pages are not merged by default. In general, merging starts when the amount of free RAM drops below 20% and stops when free RAM goes above this threshold again. It works differenly, however, depending on the policy:

• With the density policy enabled, merging works as described on server’s entire RAM, disregarding NUMA nodes.

• With the performance policy enabled, pages are merged independently within each NUMA node. This slows KSM by as many times as there are NUMA nodes on the server (e.g., by four times if there are four NUMA nodes). Unlike density, however, pages are unmerged as soon as they can fit into RAM again, leaving 20% of it free. This is done to avoid COW operations and increase server performance.

KSM is enabled by default. If you need to disable it for some reason, set KSM to false in /etc/vz/vcmmd.conf. Add the parameter if needed.

"LoadManager": {
    "Controllers": {
        "KSM": false,
        <...>
    },

Restart vcmmd to apply changes. See Restarting VCMMD.

10.1.3. Managing Host Services with VCMMD¶

You can set memory limits and guarantees managed by vcmmd for host services. The daemon will take them into account when allocating resources for virtual environments.

For example, to have a service named service managed by vcmmd, do as follows:

1. Place the service processes in /sys/fs/cgroup/memory/service.slice directory.

2. Register service.slice in vcmmd, specifying the desired guarantee and limit in bytes (you can also use K for kibibytes, M for mebibytes, or G for gibibytes). For example:

   # vcmmdctl register SRVC service.slice --guarantee 100000000 --limit 100000000 --swap 100000
   

   These settings will last until a server reboot. A way to set permanent limits is provided further in this section.

3. Activate the service:

   # vcmmdctl activate service.slice
   

List memory slices to check that the service is managed with vcmmd. For example:

# vcmmdctl list
name                                     type  active  guarantee     limit    swap
<...>
service.slice                            SRVC     yes      97656     97656      97

You can also temporarily change the service memory parameters at runtime using the vcmmdctl update command.

For example, to change the service.slice parameters, run:

# vcmmdctl update service.slice --guarantee 200000000 --limit 200000000

List memory slices to check that the parameters have been applied. For example:

# vcmmdctl list
name                                     type  active  guarantee     limit    swap
<...>
service.slice                            SRVC     yes     195312    195312     max

To unregister a service from vcmmd at runtime, use the vcmmdctl unregister command.

Guarantees and limits set using the instructions above will be removed on host reboot. You can, however, set permanent limits and guarantees for services in system and user memory slices. For example, to make sure that you can still access the host via SSH if it becomes heavily overcommitted. Do as follows:

1. If needed, dump the current values to the VCMMD configuration file:

   # vcmmdctl config -f > /etc/vz/vcmmd.conf
   

2. Specify the minimum and maximum values in bytes in /etc/vz/vcmmd.conf (-1 stands for unlimited memory). For example:

   {
     "VStorage": {
       <...>
     },
     "System": {
       "Path": "system.slice",
       "Limit": {
         "Max": -1,
         "Min": 0,
         "Share": 0.7
       },
       "Guarantee": {
         "Max": 1073741824,
         "Min": 536870912,
         "Share": 0.25
       },
       "Swap": {
         "Max": 0,
         "Min": 0,
         "Share": 0
       }
     },
     "User": {
       "Path": "user.slice",
       "Limit": {
         "Max": -1,
         "Min": 0,
         "Share": 0.7
       },
       "Guarantee": {
         "Max": 536870912,
         "Min": 268435456,
         "Share": 0.25
       },
       "Swap": {
         "Max": 0,
         "Min": 0,
         "Share": 0
       }
     }
   }
   

   Note

   The file /etc/vz/vstorage-limits.conf is deprecated and will be dropped in the coming releases.

3. Restart vcmmd to apply changes. See Restarting VCMMD.

4. Check that the parameters were applied. The actual guarantees and limits are calculated based on the minimum and maximum values specified in /etc/vz/vcmmd.conf.

   # vcmmdctl list
   name                                   type active guarantee     limit      swap
   system.slice                           SRVC    yes   1048576   5605796         0
   user.slice                             SRVC    yes    524288   5605796         0
   <...>
   

   Note

   This command outputs values in kibibytes.

10.1.4. Managing Virtuozzo Storage Services with VCMMD¶

On servers running Virtuozzo Storage, vstorage.slice/vstorage-services.slice is automatically created in the memory cgroup on server start.

If, however, you manually deployed Virtuozzo Storage on top of Virtuozzo Hybrid Server and started the vstorage* services without rebooting the server, restart vcmmd to create this memory slice. Consult Restarting VCMMD.

List memory slices to check that Virtuozzo Storage services are managed by VCMMD. For example:

# vcmmdctl list
name                                     type  active  guarantee     limit    swap
<...>
vstorage.slice/vstorage-services.slice   SRVC     yes    4194304 103854973       0

You can temporarily change memory limits and guarantees for Virtuozzo Storage services at runtime using the vcmmdctl update command. For example:

# vcmmdctl update vstorage.slice/vstorage-services.slice --guarantee 100000000 --limit 100000000

The parameters will be reset to the default values on the next vcmmd restart or node reboot.

To configure limits and guarantees for Virtuozzo Storage services permanently, do as follows:

1. If needed, dump the current values to the VCMMD configuration file:

   # vcmmdctl config -f > /etc/vz/vcmmd.conf
   

2. Specify the values in bytes in /etc/vz/vcmmd.conf (-1 stands for unlimited memory). For example:

   {
     "VStorage": {
       "Path": "vstorage.slice/vstorage-services.slice",
       "Limit": {
         "Max": -1,
         "Min": 0,
         "Share": 0.7
       },
       "Guarantee": {
         "Max": 1073741824,
         "Min": 536870912,
         "Share": 0.25
       },
       "Swap": {
         "Max": 0,
         "Min": 0,
         "Share": 0
       }
     },
     "System": {
       <...>
     },
     "User": {
       <...>
     }
   }
   

3. Restart vcmmd to apply changes. See Restarting VCMMD.

List memory slices to check that the parameters were applied. The actual guarantees and limits are calculated based on the minimum and maximum values specified in /etc/vz/vcmmd.conf.

# vcmmdctl list
name                                   type active guarantee     limit      swap
<...>
vstorage.slice/vstorage-services.slice   SRVC    yes   1048576   5605796         0

If you use large CS journals, you may also attempt to improve storage write performance by tweaking the VCMMD configuration.

Initially, storage services can use two thirds of host’s memory. As soon as you create a virtual environment, the memory available to storage services becomes limited to 512MB per CS, which may throttle their performance. You can, however, let these services use more of host’s RAM by adjusting the StorageCacheLimitTotal value. In general, set it to at least 4 GB per CS with a journal.

For example, if you have 10 CSes with journals, set the parameter to 40 GiB by adding it to the LoadManager section of /etc/vz/vcmmd.conf:

"LoadManager": {
    "Controllers": {
        <...>
        "StorageCacheLimitTotal": 42949672960,
        <...>
    },
    <...>
},

Restart vcmmd to apply changes. See Restarting VCMMD.

Check that the changes were applied:

# vcmmdctl config | grep StorageCacheLimitTotal
        "StorageCacheLimitTotal": 42949672960,

Other parameters that you may see in the Controllers section of /etc/vz/vcmmd.conf include:

• NUMA, which switches balancing of virtual environments between NUMA nodes. It is only used when the performance policy is enabled.

• StoragePolicy, which is enabled if the node is in a Virtuozzo Storage cluster. It adjusts swappiness, manages memory limits for Virtuozzo Storage services mentioned above, and protects Virtuozzo Storage services from the OOM killer.

Both of these parameters are set automatically and need not be tweaked.