2.1. System Configuration Files

The table below lists the configuration files available in Virtuozzo Hybrid Server 7. Most files are located in the /etc directory on a hardware node. If a configuration file is stored in a place other than the hardware node, its exact location is specified.

Name

Description

/etc/vz/vz.conf

Global configuration file. This file keeps system-wide settings, such as the default location of templates and global network settings.

/etc/vz/conf/<CT_name>.conf

Private configuration file of a container with the name <CT_name>. This file keeps container-specific settings: resource management parameters, the location of its private area, IP address, and so on.

/etc/vz/conf/ve-<name>.conf-sample

Sample files containing a number of default container configurations. Some pre-created samples file are shipped with Virtuozzo Hybrid Server (e.g., basic and confixx), but you can also create your own samples to meet your demands.

/usr/libexec/libvzctl/dists/ <distribution_name>.conf

Linux distribution configuration files. These files define what scripts should be run when you perform specific operations with containers (e.g., when you set a new IP address for a container). The scripts differ from Virtuozzo Hybrid Server action scripts and depend on the Linux version a particular container is running.

/etc/vz/pfcache.conf

Configuration file used by the pfcache utility to manage memory and IOPS deduplication.

/etc/vz/conf/networks_classes

Configuration file defining the network classes for traffic shaping and bandwidth management.

/etc/sysctl.d/*

Configuration files with kernel parameters required for VMs and containers to work.

/etc/vztt/vztt.conf

Configuration file used by the vzpkg utility to manage OS and application EZ templates.

/etc/vz/tools-update.conf

Configuration file used by the vz-guest-tools-updater script to manage automatic Virtuozzo Hybrid Server tools updating.

/etc/shaman/shaman.conf

Local configuration file used by the shaman and shaman-monitor utilities to manage and monitor high availability clusters.

/vstorage/<cluster_name>/.drs/config

Global configuration file used by the pdrs daemon to configure the DRS resource relocation mode for high availability clusters.

2.1.1. Global System Configuration File

Virtuozzo Hybrid Server keeps its system wide configuration parameters in the /etc/vz/vz.conf configuration file. This file is in shell format. Keep in mind that Virtuozzo Hybrid Server scripts source this file - thus, shell commands in this file will cause system to execute them under root account. Parameters in this file are presented in the form PARAMETER="value". Logically all the parameters belong to the following groups: global parameters, logging, disk quotas, template, network traffic, and container default parameters. Below is the description of all the parameters defined in this version of Virtuozzo Hybrid Server.

Name

Description

Default Value

VIRTUOZZO

Can be yes or no. Virtuozzo Hybrid Server System V startup script checks this parameter. If set to no, then Virtuozzo Hybrid Server modules are not loaded. You might set it to “no” if you want to perform system maintenance and do not want to bring up all containers on the server.

yes

HTTP_PROXY

Specifies either the hostname or the IP address of the HTTP proxy server. After setting this parameter and in case you use an HTTP proxy server for handling all HTTP requests, the Virtuozzo Hybrid Server utilities communicating with the outer world through HTTP will use this server for managing all your HTTP messages.

n/a

ACTIONLOGDIR

This is the directory where prlctl keeps a log of its actions in the format suitable for Virtuozzo Hybrid Server statistics daemon hwcoll.

/vz/actionlog

LOCKDIR

Actions on a container should be serialized, since two simultaneous operations on the same container may break its consistency. Virtuozzo Hybrid Server keeps lock files in this directory in order to serialize access to one container.

/vz/lock

VEFSTYPE

File system to use when caching OS templates:

  • ext4,

  • simfs.

ext4

IPV6

Defines whether the IPv6 support is enabled on the hardware node.

yes

GOLDEN_IMAGE

Enables (yes) or disables (no) embedding application templates into OS EZ template cache prior to creating containers based on this cache.

yes

PFCACHE

Path to the memory and IOPS deduplication cache with common container files.

/vz/pfcache

PFCACHE_IMAGE

Path to the private area of the memory and IOPS deduplication cache.

/vz/pfcache.hdd

PFCACHE_IMAGE_SIZE

Image size (in 1KB blocks) of the memory and IOPS deduplication cache.

10485760

PFCACHE_INCLUDES

Directories for which memory and IOPS deduplication is enabled by default.

bin lib lib64 opt sbin usr

VZ_TOOLS_BCID

Enables limits for the backup, restore, and migration operations.

 

VZ_TOOLS_IOLIMIT

Sets the disk I/O limit for the backup and migration operations, in bytes per second. Not set by default.

 

ON_VE_FSERROR

The action to be performed on a container if a file system error occurs. Currenly, the only available action is forced container stop. To enable, set the parameter to stop.

n/a

SERVER_UUID

(Virtuozzo Hybrid Server 7.0.6 and newer) Unique server identifier that is randomly generated and set on the vz service start. It is used to identify host ownership of containers on shared storage and provide access to their disks.

Note

If you want to deploy multiple servers from a backup, make sure this parameter is not in the source server configuration file. Otherwise, restored servers will have the same SERVER_UUID and their containers host ownership identification will fail.

n/a

DEF_OSTEMPLATE

An OS template for newly created containers.

.centos-7 for previous versions and .almalinux-8 for new VHS versions.

Logging parameters:

Name

Description

Default Value

LOGGING

This parameter defines whether prlctl should log its actions.

yes

LOGFILE

File where libvzctl logs the actions of programs linked to this library.

/var/log/vzctl.log

LOG_LEVEL

Logging verbosity, from 0 to 10 (higher is more verbose).

0

Disk quota parameters:

Name

Description

Default Value

DISK_QUOTA

Enables or disables disk quotas for containers. If set to no then disk space accounting will be disabled.

yes

Network traffic parameters:

Name

Description

Default Value

TRAFFIC_SHAPING

Traffic shaping allows you to limit the bandwidth consumed by containers for outgoing traffic. If it is set to “yes”, then limitations will be turned on. If you want to use this feature, TRAFFIC_ACCOUNTING should be set to yes as well.

no

BANDWIDTH

This is the list of network interfaces on which we want to shape the traffic and their speed in the form of “dev:rate”. The rate is measured in Kbps. If you want to shape traffic on more than one interface, set this parameter to dev1:rate1 dev2:rate2. For example, for two 100 Mbps Ethernet cards, set it to enp0s5:100000 enp0s6:100000.

enp0s5:100000

TOTALRATE

This parameter sets the size of the bandwidth pool for all containers. It is the upper limit for the bandwidth available to all your containers and is specified in the form of “dev:class:rate”. The rate is measured in Kbps. Containers can consume bandwidth up to this limit in addition to the limit specified by the RATE parameter. Default value corresponds to 4 Mbps limit for the Class 1 containers.

*:1:4000

RATE

This parameter is the default bandwidth guaranteed to a container for outgoing traffic if the container configuration file does not explicitly specify a different value. This value is in the same format as TOTALRATE (“dev:class:rate”). The rate is measured in Kbps. Note that 8 Kbps, offered by the default configuration, is the guarantee and the container cannot consume less than this value and more than the sum of this value and TOTALRATE.

*:1:8

RATEMPU

This optional parameter (where MPU stands for “minimum packet unit”) limits the packet rate by making packets smaller than MPU in size consume HTB tokens. With it, small packets can be accounted as larger ones and limited by TOTALRATE and RATE parameters. Approximately, the maximum packets per second rate can be calculated as TOTALRATE / RATEMPU. The parameter is specified in the form “dev:class:MPU per packet”. If the part “MPU per packet” is omitted, the default value of 1000 bytes is used.

*:1:1000

Template parameters:

Name

Description

Default Value

TEMPLATE

This is the directory where to find templates. It is not recommended to redefine this option since all Virtuozzo Hybrid Server templates use the default directory.

/vz/template

Container default parameters:

Name

Description

Default Value

VE_ROOT

The mount point for container’s root. Must contain the literal string $VEID that will be substituted with the actual container UUID.

/vz/root/$VEID

VE_PRIVATE

The directory where all the files and directories specific to the container are stored. Must contain the literal string $VEID that will be substituted with the actual container UUID.

/vz/private/$VEID

CONFIGFILE

The default configuration file sample to be used for the container creation; it may be overridden with the --config option of the prlctl create command.

basic

DEF_OSTEMPLATE

The default OS template to be used for the container creation.

centos-7 for previous versions. almalinux-8 for new VHS versions.

VE_ENVIRONMENT

Additional environment variables to be passed to the container init process. Should be provided as any number of <name>=<value> pairs separated by spaces.

 

2.1.2. Container Configuration File

Each container has its own configuration file, which is stored in the /etc/vz/conf directory and has a name like <CT_name>.conf. This file has the same format as the global configuration file. The settings specified in this file can be subdivided into the following categories:

  • miscellaneous,

  • resource management parameters,

  • networking.

2.1.2.1. Miscellaneous Parameters

The table below list the miscellaneous parameters you can set in the configuration file of a container:

Name

Description

VERSION

Specifies the Virtuozzo Hybrid Server version the configuration file applies to. 2 relates to Virtuozzo Hybrid Server version 4 and newer.

ONBOOT

Specifies whether the container should be started automatically on system startup. Virtuozzo Hybrid Server automatically starts all containers that have this parameter set to “yes” upon startup.

ALLOWREBOOT

Specifies whether the container may be restarted with the reboot command run from inside. If omitted or set to yes, restarting is allowed.

OSTEMPLATE

The name of the OS template that was used for creating the container. You do not have to change this parameter; prlctl will set it for you upon calling the prlctl create command (or using the defaults from the global configuration file). The . symbol before the OS template name, if specified, indicates that this is an EZ OS template.

TEMPLATES

In a configuration file of an existing container, this parameter lists application templates installed with the prlctl create or vzpkg install commands. In this case you should not modify it, because it is used by template management utilities to track installation history. This parameter is omitted if no templates have been installed to the container.

VE_ROOT

Overrides the VE_ROOT parameter from the global configuration file.

VE_PRIVATE

Overrides the VE_PRIVATE parameter from the global configuration file.

VE_ENVIRONMENT

Overrides the VE_ENVIRONMENT parameter from the global configuration file.

TECHNOLOGIES

Determines a set of technologies which should be provided by the Virtuozzo Hybrid Server kernel for container operation. Currently, this parameter can contain the information about the following technologies:

  • The system architecture of the container (x86, x86_64, or i64).

  • Whether the container is based on the OS template supporting the Native POSIX Thread Library (NPTL). In this case, the nptl entry is specified as the value of this parameter.

  • Whether the OS EZ template the container is based on requires the sysfs file system support (e.g., the OS EZ template for SUSE Linux Enterprise 10).

DISABLED

If set to yes, disables the container making it impossible to start the container once it was stopped. You can start the disabled container after setting the value of this parameter to no.

DESCRIPTION

Sets the description for the container.

Note

You are allowed to use only symbols in the A-z and 0-9 ranges in your descriptions.

NAME

Container name that can be used to refer to said container in commands. Names must be alphanumeric and may contain the characters \, -, _. Names with white spaces must be enclosed in quotation marks.

ORIGIN_SAMPLE

The configuration sample the container was based on when created.

CONFIG_CUSTOMIZED

Indicates whether any of the container configuration parameters have been modified as regards its original configuration sample. If this parameter is omitted, its value is considered as no.

UUID

The container unique identifier. This identifier is used by certain Virtuozzo Hybrid Server utilities during their execution.

2.1.2.2. Resource Management Parameters

All resource management parameters can be subdivided into the CPU, disk, system, and VSwap categories for your convenience. Any parameter can be set with the prlctl set command and the corresponding option name (in the lower case, e.g., --cpuunits for CPUUNITS, etc.). See Managing Containers for more details. The Typical value column, if present, specifies a range of reasonable parameter values for different applications, from light to huge heavy loaded containers. If the barrier and limit fields are in use, ranges for both thresholds are given.

CPU parameters:

Parameter

Description

Typical value

CPUUNITS

CPU weight. This is a positive integer number that defines how much CPU time the container can get as compared to the other virtual machines and containers running on the server. The larger the number, the more CPU time the container can receive. Possible values range from 8 to 500000. If this parameter is not set, the default value of 1000 is used.

250...1000

CPULIMIT, CPULIMIT_MHZ

CPU limit, in per cent (CPULIMIT) or megahertz (CPULIMIT_MHZ), the container is not allowed to exceed. The parameter is not set for newly created containers; so they can consume all free CPU power of the server. When setting this parameter in per cent, keep in mind that one CPU core makes up 100%. So if the server has 4 CPU cores, the total CPU power will equal 400%.

 

CPUS

Number of CPU cores defining the CPU limit for a container. The limit is calculated by multiplying the power of one CPU core by the number of the specified CPU cores. This option also defines the number of CPUs shown to users from inside a container. This parameter is not set for newly created containers; so they can consume all free CPU power of the server.

 

CPUMASK

The CPU affinity mask defining which CPUs on the Node can be used to handle the processes running in the container. The CPU mask can be specified as both separate CPU index numbers (1,2,3) and CPU ranges (2-4,5-7).

 

NODEMASK

The NUMA node mask defining a NUMA node to bind the container to. Once you set the mask, the processes running in the container will be executed only on the CPUs that belong to the specified NUMA node.

 

Disk parameters:

Parameter

Description

Typical value

DISKSPACE

Total size of disk space that can be consumed by the container, in 1 KB blocks.

QUOTAUGIDLIMIT

This parameter enables (if set to a value other than 0) or disables (if set to 0) per-user and per-group quotas for further management with the standard Linux quota utility. Enabling per-user and per-group quotas for a container requires restarting the container.

0...N

IOPRIO

The container priority for disk I/O operations. The higher the priority, the more time the container has for writing to and reading from the disk. The default container priority is 4.

0-7

IOPSLIMIT

The maximum number of disk input and output operations per second a container is allowed to perform. By default, any newly created container does not have the IOPS limit set and can perform so many disk I/O operations per second as necessary.

 

IOLIMIT

The bandwidth a container is allowed to use for its disk input and output (I/O) operations. By default, the limit is set in megabytes per second. However, you can use the following suffixes to use other measurement units:

  • G - sets the limit in gigabytes per second.

  • K - sets the limit in kilobytes per second.

  • B - sets the limit in bytes per second.

In the current version of Virtuozzo Hybrid Server, the maximum I/O bandwidth limit you can set for a container is 2 GB per second.

The default I/O bandwidth limit for all newly created containers is set to 0, which means that no limits are applied to any containers.

 

System parameters:

Parameter

Description

Typical value

NUMPROC

Number of processes and threads allowed. Upon hitting this limit, container will not be able to start a new process or thread.

40...400

AVNUMPROC

Number of processes expected to run in the container on average. This is informational parameter used to ensure configuration correctness.

0...NUMPROC

VMGUARPAGES

Memory allocation guarantee, in pages. Applications are guaranteed to be able to allocate memory while the amount of memory accounted as privvmpages does not exceed the configured barrier of the vmguarpages parameter. Above the barrier, memory allocation is not guaranteed and may fail in case of overall memory shortage.

1725...107520

LOCKEDPAGES

Memory not allowed to be swapped out (locked with the mlock() system call), in pages (one page is 4 KB).

4...4096

SHMPAGES

Total size of shared memory (including IPC, shared anonymous mappings and tmpfs objects), allocated by processes of a particular container, in pages.

512...16384

PRIVVMPAGES

Size of private (or potentially private) memory, allocated by an application. Memory that is always shared among different applications is not included in this resource parameter.

NUMFILE

Number of files opened by all container processes.

512...8192

NUMFLOCK

Number of file locks created by all container processes.

50...200-60...220

NUMPTY

Number of pseudo-terminals. For example, the ssh session, screen, the xterm application consumes pseudo-terminal resources.

4...64

NUMSIGINFO

Number of siginfo structures (essentially this parameter limits the size of signal delivery queue).

256...512

PHYSPAGES

Total size of RAM used by processes. This parameter is used for accounting purposes only. It shows the usage of RAM by the container. For memory pages used by several different containers (mappings of shared libraries, for example), only a fraction of a page is charged to each container. The sum of the physpages for all containers corresponds to the total number of pages used in the system by all accounted users.

Not limited

NUMIPTENT

The number of IP packet filtering entries.

12...128

VSwap parameters:

Parameter

Description

Typical value

PHYSPAGES

Amount of RAM that can be used by the processes of a container, in 4KB pages.

 

SWAP

Amount of swap space that can be used by the container for swapping out memory once the RAM is exceeded, in 4KB pages.

 

VM_OVERCOMMIT

Memory overcommit factor that defines the memory allocation limit for a container. The limit is calculated as (PHYSPAGES + SWAP) * factor.

Not limited

2.1.2.3. Networking Parameters

Network-related parameters allow you to set bandwidth management parameters, hostname and IP addresses that a container can use, and other parameters.

Name

Description

HOSTNAME

If this parameter is specified, then prlctl will set the hostname to its value upon the next container start. This parameter can be omitted. In this case, the container administrator should configure the hostname manually.

IP_ADDRESS

This is the list of IP addresses, which can be used on container network interfaces. This list is an argument of the container start call and it is impossible to assign IP address from inside the container if the address is not on the list. Any IP address assigned from within the container will be visible only within the container.

NAMESERVER

The IP address of the DNS server the container is supposed to use. More than one server can be specified in the space-separated format.

SEARCHDOMAIN

DNS search domains for the container. More than one domain can be specified.

NETDEV

The names of physical network adapters that have been moved from the server to the given container.

NETFILTER

Indicates which iptables modules are allowed for the container. If some of the allowed modules are not loaded on the destination Hardware Node after migration or restoration from backup, they will be automatically loaded on the migrated or restored container start. The following modes are available:

  • disabled: none.

  • stateless: (default) all modules except conntrack and NAT-related.

  • stateful: all modules except NAT-related.

  • full: all modules.

NETIF

Specifies a number of parameters for the virtual network adapters existing inside the container. These parameters include:

  • ifname: the name of the veth virtual Ethernet interface inside the container.

  • mac: the MAC address assigned to the veth virtual Ethernet interface inside the container.

  • host_mac: the MAC address assigned to the veth virtual Ethernet interface on the server.

  • network: the name of the virtual network where the veth virtual network adapter is included.

  • ip: the IP address(es) assigned to the veth virtual network adapter.

RATE

If traffic shaping is turned on, then this parameter specifies bandwidth guarantee, in Kbps, for the container. The parameter should be set in the form of dev:class:rate.

RATEBOUND

If set to yes, the bandwidth guarantee is also the limit for the container, and the container cannot borrow the bandwidth from the TOTALRATE bandwidth pool.

2.1.3. Linux Distribution Configuration Files

Some Virtuozzo Hybrid Server tools (e.g., prlctl) need to run special scripts inside a container to perform certain operations on it. However, carrying out one and the same operation inside containers running different Linux versions may require execution of different actions. This may be caused by the fact that different Linux distributions store files in different locations, use different commands to complete one and the same task, and so on. To distinguish between containers running different Linux versions and to determine what scripts should be executed while performing the relevant container-related operations, Virtuozzo Hybrid Server uses special distribution configuration files located in the /usr/libexec/libvzctl/dists directory on the server.

There are a number of distribution configuration files shipped with Virtuozzo Hybrid Server by default (centos.conf, fedora-core.conf, gentoo.conf, etc.). To view all configuration files available on your Virtuozzo Hybrid Server, you can go to the /usr/libexec/libvzctl/dists directory and issue the ls command. The distribution configuration files will be displayed in the form of <Linux_distribution>-<version>.conf where <Linux_distribution> and <version> denote the name of the Linux distribution and its version, respectively (e.g., centos-7.conf).

Any distribution configuration file consists of a number of entries in the form of <parameter_name>=<script_name> where <parameter_name> denotes the name of the parameter defining the operation when the script in the right part of the entry is to be executed and <script_name> is the name of the script to be run on performing the operation defined by the parameter in the left part of the entry. In the current version of Virtuozzo Hybrid Server, the following parameters are used to define what scripts should be executed for the corresponding Linux version a container is running:

  • ADD_IP: the script specified as the value of this parameter has the default name of <distribution_name>-add_ip.sh and is used to configure the network settings during the container startup and the IP address(es) assignment. The script is launched inside the container on executing the following commands:

    # prlctl start <CT_name>
    # prlctl set <CT_name> --ipadd <IP_address>
    # prlctl set <CT_name> --ipadd <IP_address> --ipdel all
    
  • DEL_IP: the script specified as the value of this parameter has the default name of <distribution_name>-del_ip.sh and is used to delete an existing IP address from the container. The script is launched inside the container on executing the following commands:

    # prlctl set <CT_name> --ipdel <IP_address>
    # prlctl set <CT_name> --ipdel all
    
  • SET_HOSTNAME: the script specified as the value of this parameter has the default name of <distribution_name>-set_hostname.sh and is used to configure the hostname of the container. The script is launched inside the container on executing the following command:

    # prlctl set <CT_name> --hostname <name>
    
  • SET_DNS: the script specified as the value of this parameter has the default name of <distribution_name>-set_dns.sh and is used to configure DNS parameters in the /etc/resolv.conf file. The script is launched inside the container on executing the following command:

    # prlctl set <CT_name> --searchdomain <domain> --nameserver <IP_address>
    
  • SET_USERPASS: the script specified as the value of this parameter has the default name of <distribution_name>-set_userpass.sh and is used to add a new user or change the current password. The script is launched inside the container on executing the following command:

    # prlctl set <CT_name> --userpasswd <user>:<passwd>
    
  • SET_UGID_QUOTA: the script specified as the value of this parameter has the default name of <distribution_name>-set_ugid_quota.sh and is used to set up per-user/group quota. The script is launched inside the container on executing the following command:

    # prlctl set <CT_name> --quotaugidlimit <num>
    
  • POST_CREATE: the script specified as the value of this parameter has the default name of <distribution_name>-postcreate.sh and is used to perform certain tasks (e.g., to modify the crontab files) after the container creation. This script is launched on the server on executing the following command:

    # prlctl create <CT_name>
    

The scripts specified in distribution configuration files are located in the /usr/libexec/libvzctl/dists/scripts directory on the server and executed on performing the aforementioned operations on the containers. After an operation has been initiated, the prlctl utility turns to the corresponding container configuration file, looks for the value of the DISTRIBUTION variable or, if the latter is not present, of the OSTEMPLATE variable in this file, and defines on their basis what Linux version the given container is running. After that, prlctl reads the corresponding configuration file for the determined Linux version from the /usr/libexec/libvzctl/dists/ directory and executes the scripts specified in this file.

Note

If no distribution is specified as the value of the DISTRIBUTION and OSTEMPLATE variables in the container configuration file or no configuration file for the given Linux version was found in the /usr/libexec/libvzctl/dists directory, the default file from this directory is used.

2.1.4. Memory and IOPS Deduplication Configuration File

Contained in the /etc/vz/pfcache.conf file, memory and IOPS deduplication parameters allow you to tailor cache behavior and performance to your needs.

Name

Description

Default Value

COUNT

The minimum number of file copies required for the file to become cacheable. Copies may exist in the same container or different containers.

2

MINSIZE

Minimal cacheable file size, bytes. Files smaller than this value will not be cached.

0

MAXSIZE

Maximal cacheable file size, bytes. Files larger than this value will not be cached.

2147483648

TIMEOUT

Time between caching attempts, seconds.

5

PFCACHE_IOLIMIT

Memory and IOPS deduplication cache IO bandwidth limit, bps. Unlimited by default.

 

PFCACHE_IOPSLIMIT

Memory and IOPS deduplication cache IOPS limit. Unlimited by default.

 

LOGLEVEL

Logging verbosity. Messages are logged in the system log file /var/log/messages.

1

PAGEMIN

The total number of memory pages used in containers:

  • 0 - Cache even files without memory pages.

  • 1 - Cache only files in use.

  • N - Cache only when the total number of memory pages in containers reaches N.

1

PURGEAHEAD

Extra cache space to free up in addition to the requested space. In per cent of the requested space. Used with the pfcache purge --size command.

20%

2.1.5. Network Classes Definition File

In Virtuozzo Hybrid Server, both traffic accounting and bandwidth management are based on network classes. The network classes’ definition file (/etc/vz/conf/networks_classes) describes network classes that Virtuozzo Hybrid Server recognizes. Currently, there can be up to 15 classes defined.

The lines in this file have the following format:

<class_ID> <IP_address>/<prefix_length>

where <class_ID> defines the network class identifier, <IP_address> defines the starting IP address, and <prefix_length> defines the subnet mask. In pair <IP_address> and <prefix_length> define the range of IP addresses for this class. There may be several lines for each class. Classes should be defined after Class 1 and represent exceptions from the “matching-everything” rule of Class 1. Class 0 has a special meaning and defines the IP ranges for which no accounting is done (this server container addresses).

The definition of class 1 is required; any class except class 1 can be omitted. However, it is recommended to define class 0 correctly as it will improve performance. For example:

# HW node VPS's networks
0 10.10.10.0/24
0 10.10.15.0/24
# all IP("local" traffic)
1 0.0.0.0/0
# class 2 - "foreign" traffic
#2 10.0.0.0/8
#2 11.0.0.0/8
# inside "foreign" network there
# is a hole with "local" traffic
#1 10.10.16.0/24

2.1.6. Offline Management Configuration Files

The offline management configuration files located in the /etc/vzredirect.d directory define various modes of container offline management by container administrators. One configuration file describes one offline management mode. In the current Virtuozzo Hybrid Server version, two files are accessible: vzpp.conf and vzpp-plesk.conf. The first file defines the container offline management by means of Power Panel, and the second one by means of the same Power Panel with an integrated Plesk control panel.

There are two parameters in each of the files.

<parameter_name>=<parameter_value>

Name

Description

Example

PORT

This port must be entered in the address line of an Internet browser after the container IP address when managing the container by means of Power Panel or the Plesk control panel.

PORT=8443

DST_VEID

The UUID of the container where the requests coming to the specified port will be redirected.

DST_VEID=1

2.1.7. vztt Configuration File

This file (/etc/vztt/vztt.conf) is the configuration file used by the vzpkg utility when managing OS and application EZ templates.

<parameter_name>=<parameter_value>

Name

Description

VZTT_PROXY

The IP address or hostname of the caching proxy server to be used by the vzpkg tool for managing OS and application EZ templates.

HTTP_PROXY

The IP address or hostname of the HTPP proxy server address, if you use this server.

HTTP_PROXY_USER

The user name used by the HTTP proxy server for your authentication.

HTTP_PROXY_PASSWORD

The password of the user specified in the HTPP_PROXY_USER parameter and used for your authentication by the HTTP proxy server.

METADATA_EXPIRE

Defines the period of time, in seconds, in the course of which the downloaded software packages in the vzpkg cache are regarded as not obsolete. During this time, the vzpkg utility searches for the EZ template packages in the local cache only (without checking the remote repositories set for EZ templates). By default, this period is set to 86400 seconds (24 hours).

EXCLUDE

List of comma-separated packages that are not to be installed or updated during the vzpkg execution. The package names should correspond to the name of real packages in the repository and can contain file globs (e.g., * and ?).

2.1.8. pcompact.conf

The /etc/vz/pcompact.conf file is used by the pcompact utility to compact virtual disks in containers.

<parameter_name>=<parameter_value>

Name

Description

THRESHOLD=<number>

Compact the virtual disk if unused space on it exceeds THRESHOLD percent of the ploop size.

DELTA=<number>

Reduce disk space to be compacted by DELTA percent of the ploop size.

DEFRAG=<yes|no>

Perform or skip file system defragmentation.

2.1.9. tools-update.conf

The file /etc/vz/tools-update.conf is used by the vz-guest-tools-updater script to manage automatic Virtuozzo Hybrid Server tools updating.

"<parameter_name>": <parameter_value>

Name

Description

"MaxVMs": <number>

Sets the maximum number of virtual machines whose Virtuozzo Hybrid Server tools can be updated simultaneously.

"InstallTools": true|false

Enables or disables automatic installation of Virtuozzo guest tools in virtual machines.

2.1.10. shaman Configuration File

/etc/shaman/shaman.conf is the local configuration file for the shaman-monitor daemon and the shaman command-line tool. It affects settings of shaman-monitor running on the particular node. At the same time, a number of cluster-wide configuration parameters used by the shaman-monitor daemon are defined in the global configuration file shaman-config. You can obtain these parameters by running the shaman get-config command and modify them with shaman set-config.

<parameter_name>=<parameter_value>

Local parameters:

Name

Description

CLUSTER_NAME

Set the name of the cluster for shaman to operate on.

LOG_LEVEL

Set the verbosity level for printed messages:

  • 0: print error messages only;

  • 1: print warning messages in addition to error messages;

  • 2 (default): print information messages in addition to errors and warnings;

  • 4: print debug diagnostic messages in addition to messages from levels 0-2.

Global parameters:

Name

Description

LOCK_TIMEOUT

Set the timeout for shaman-monitor operations (e.g., electing a new master or deciding that a slave node is down). This parameter helps avoid situations when shaman-monitor performs a cluster-related operation if someone pulls out the network cable for just a couple of seconds (not longer than the half of the LOCK_TIMEOUT value).

The default value is 60 seconds.

The specified value is added to the value of other timeout-related parameters (see below).

LEASE_CHECK_TIMEOUT_FOR_MASTER

Set the timeout for electing a new master node when the original master node or the shaman-monitor daemon fails, or high availability support gets disabled.

The default value is 10 seconds.

LEASE_CHECK_TIMEOUT_FOR_SLAVE

Set the timeout after which the master node will consider a slave node as broken if this node and the shaman-monitor daemon get down, or high availability support gets disabled.

The default value is 10 seconds.

LEASE_LOST_ACTION

Defines the action to perform if shaman-monitor loses connection to cluster.

This may happen when a node goes online after having been disconnected from network for more than LOCK_TIMEOUT seconds. In this case, the watchdog timer has not expired yet, but the cluster is already unavailable, because the master node has prohibited access to the cluster until the node is rebooted.

Available values are crash, halt, reboot, and none (do nothing).

CLUSTER_MOUNTPOINT_DEAD_ACTION

Defines the action to perform when shaman-monitor detects that the cluster mount point is no longer functioning properly for some reason.

The supported actions are crash, halt, reboot, none.

RELOCATION_SKIP_THRESHOLD

Sets the threshold for the number of simultaneously crashed nodes.

If the number of simultaneously crashed nodes becomes greater than or equal to the threshold, the master stops relocating resources from the crashed nodes. When the number of simultaneously crashed nodes drops below the threshold, the master automatically resumes relocating resources from the crashed nodes. The threshold can be useful when multiple nodes are being rebooted at the same time. Without it, the master would start relocating resources from all the rebooting nodes.

The threshold is set to 3 by default and must be 2 or greater. For clusters with only 3 nodes, the threshold is automatically set to 2.

POOL_CHECK_TIMEOUT

Set the interval for shaman-monitor to check for the resources scheduled for relocation.

The default value is 30 seconds.

RESOURCE_RELOCATION_MODE

Defines a sequence of algorithms (modes) used for resource relocation on hardware node failure. At least one mode must be specified. Multiple modes must be separated with commas. On hardware node failure, relocation using the first specified mode is attempted. If unsuccessful, the next specified mode is attempted and so on. If relocation using the last specified mode is unsuccessful, the resources are left on the failed hardware node. The following resource relocation modes are supported:

  • round-robin: Each resource from the failed hardware node is relocated to another node, which is chosen using the round-robin algorithm. In general, resources are relocated to different hardware nodes.

  • spare: All resources from the failed hardware node are relocated to a ‘spare’ node. A spare node is a hardware node, which is registered in the cluster and has no resources stored on it.

  • drs: All resources from the failed hardware node are relocated using an external DRS daemon.

The default sequence is drs, round-robin.

WATCHDOG_TIMEOUT

Set the interval for the watchdog timer.

The watchdog timer is responsible for performing the action defined in WATCHDOG_ACTION (see below) if shaman-monitor crashes or hangs up. The shaman-monitor daemon activates the watchdog timer on its start-up and periodically resets it to the specified value. If something goes wrong with shaman-monitor so that it fails to reset the timer, the watchdog timer counts down until it reaches zero and performs the defined action. Setting the interval to zero disables the watchdog timer. Minimal watchdog timer interval that could be set is 10 seconds.

The default value is 120 seconds.

WATCHDOG_ACTION

Defines a sequence of actions to perform after the watchdog timer expires (happens when shaman-monitor crashes or hangs up).

When the watchdog timer expires, the first specified action is attempted. If unsuccessful, the next specified action is attempted and so on. If the last specified action is unsuccessful, then the action specified in the /sys/kernel/watchdog_action file is performed.

At least one action must be specified. Multiple actions must be separated with commas. Available actions are listed in the /sys/kernel/watchdog_available_actions file. The default sequence is netfilter, reboot.

2.1.11. pdrs Configuration File

The file /vstorage/<cluster_name>/.drs/config is used by the pdrs daemon to configure the DRS resource relocation mode for high availability clusters.

<parameter_name>=<parameter_value>

Name

Description

SNMP_PORT

Specifies a UDP listen port for SNMP traps.

The default port is 33333.

SNMP_TRAP_PERIOD

Sets a period between SNMP traps.

The default value is 60.

SNMP_TRAP_NUMBER

Sets the number of SNMP traps.

The default value is 20.

DISP_CONN_TIMEOUT

Sets a dispatcher connection timeout.

The default value is 600.

MASTER_LEASE_TIMEOUT

Sets a master node connection timeout, after which said node will lose its master status in the cluster.

The default value is 60.

MEMPOL_CT_RAM_OVERCOMMIT

Sets an overcommit coefficient used to find a suitable node according to its available RAM capacity while relocating containers from failed nodes to healthy ones. The amount of RAM required for a container on a healthy node can be calculated by dividing the container RAM size by its overcommit coefficient. The value is specified as a decimal fraction.

The default value is 2.0.

MEMPOL_VM_RAM_OVERCOMMIT

Sets an overcommit coefficient used to find a suitable node according to its available RAM capacity while relocating virtual machines from failed nodes to healthy ones. The amount of RAM required for a virtual machine on a healthy node can be calculated by dividing the VM RAM size by its overcommit coefficient. The value is specified as a decimal fraction.

The default value is 1.0.