7.9. Configuring Memory Policy for Storage Services

You can configure memory limits and guarantees for storage services at runtime using the vinfra memory-policy vstorage-services commands. You can do this for the entire cluster or a specific node.

The following memory parameters can be configured manually:

  • memory guarantee
  • swap size
  • page cache (which, in turn, is set using cache ratio, minimum, and maximum)

Page cache is calculated according to the following formula:

$PAGE_CACHE = minimum <= ratio * $TOTAL_MEMORY <= maximum

The minimum and maximum values are hard limits that are applied if the ratio * $TOTAL_MEMORY value is outside these limits.

To better understand how page cache size is calculated, consider the following examples:

Table 7.9.1 Page cache examples
 

Example 1

(cache size is within limits)

Example 2

(cache size equals minimum)

Example 3

(cache size equals maximum)

Total memory 4 GiB 4 GiB 4 GiB
Cache ratio 0.5 0.1 0.9
Cache minimum 1 GiB 2 GiB 1 GiB
Cache maximum 3 GiB 3 GiB 3 GiB
Cache size 2 GiB 2 GiB 3 GiB

If memory parameters are set both per node and per cluster, the per-node ones are applied. If no memory parameters are configured manually, the memory management is performed automatically by the vcmmd daemon as follows:

  • Each CS (e.g., storage disk) requires 512 MiB of RAM for page cache.
  • The page cache minimum is 1 GiB.
  • If the total memory is less than 48 GiB, the page cache maximum is calculated as two-thirds of it.
  • If the total memory is greater than 48 GiB, the page cache maximum is 32 GiB.

To check the current memory parameters for storage services set by vcmmd, run

# vcmmdctl list
name                                    type active guarantee     limit   swap     cache
<...>
vstorage.slice/vstorage-services.slice  SRVC    yes   1310720  24522132      0   1048576

7.9.1. vinfra memory-policy vstorage-services per-cluster change

Change per-cluster memory parameters:

usage: vinfra memory-policy vstorage-services per-cluster change
       [--guarantee <guarantee>] [--swap <swap>] [--cache-ratio <cache-ratio>
       --cache-minimum <cache-minimum> --cache-maximum <cache-maximum>]
--guarantee <guarantee>
Guarantee, in bytes
--swap <swap>
Swap size, in bytes, or -1 if unlimited
--cache-ratio <cache-ratio>
Cache ratio from 0 to 1 inclusive
--cache-minimum <cache-minimum>
Minimum cache, in bytes
--cache-maximum <cache-maximum>
Maximum cache, in bytes

Example:

# vinfra memory-policy vstorage-services per-cluster change --guarantee 8796093022208
--swap 1099511627776 --cache-ratio 0.5 --cache-minimum 1099511627776
--cache-maximum 3298534883328
+-----------+------------------------+
| Field     | Value                  |
+-----------+------------------------+
| cache     | maximum: 3298534883328 |
|           | minimum: 1099511627776 |
|           | ratio: 0.5             |
| guarantee | 8796093022208          |
| swap      | 1099511627776          |
+-----------+------------------------+

This command sets the storage services memory parameters for all nodes in the storage cluster as follows:

  • the memory guarantee to 8 GB
  • the swap size to 1 GB, and
  • the page cache limits: the minimum to 1 GB, the maximum to 3 GB, and the cache ratio to 0.5.

7.9.2. vinfra memory-policy vstorage-services per-cluster show

Show per-cluster memory parameters:

usage: vinfra memory-policy vstorage-services per-cluster show

Example:

# vinfra memory-policy vstorage-services per-cluster show
+-----------+------------------------+
| Field     | Value                  |
+-----------+------------------------+
| cache     | maximum: 3298534883328 |
|           | minimum: 1099511627776 |
|           | ratio: 0.5             |
| guarantee | 8796093022208          |
| swap      | 1099511627776          |
+-----------+------------------------+

This command lists the storage services memory parameters for all nodes in the storage cluster.

7.9.3. vinfra memory-policy vstorage-services per-cluster reset

Reset per-cluster parameters to default:

usage: vinfra memory-policy vstorage-services per-cluster reset [--guarantee]
       [--swap] [--cache]
--guarantee
Reset only the guarantee.
--swap
Reset only the swap size.
--cache
Reset only cache values.

Example:

# vinfra memory-policy vstorage-services per-cluster reset --cache
+-----------+---------------+
| Field     | Value         |
+-----------+---------------+
| cache     |               |
| guarantee | 8796093022208 |
| swap      | 1099511627776 |
+-----------+---------------+

This command resets the manually configured page cache limits to default for all nodes in the storage cluster.

7.9.4. vinfra memory-policy vstorage-services per-node change

Change per-node memory parameters:

usage: vinfra memory-policy vstorage-services per-node change [--guarantee <guarantee>]
       [--swap <swap>] [--cache-ratio <cache-ratio> --cache-minimum <cache-minimum>
       --cache-maximum <cache-maximum>] --node <node>
--guarantee <guarantee>
Guarantee, in bytes
--swap <swap>
Swap size, in bytes, or -1 if unlimited
--cache-ratio <cache-ratio>
Cache ratio from 0 to 1 inclusive
--cache-minimum <cache-minimum>
Minimum cache, in bytes
--cache-maximum <cache-maximum>
Maximum cache, in bytes
--node <node>
Node ID or hostname

Example:

# vinfra memory-policy vstorage-services per-node change --guarantee 3298534883328
--swap 1099511627776 --cache-ratio 0.5 --cache-minimum 1099511627776
--cache-maximum 3298534883328 --node 7ffa9540-5a20-41d1-b203-e3f349d62565
+-----------+------------------------+
| Field     | Value                  |
+-----------+------------------------+
| cache     | maximum: 3298534883328 |
|           | minimum: 1099511627776 |
|           | ratio: 0.5             |
| guarantee |                        |
| swap      | 1099511627776          |
+-----------+------------------------+

This command sets the storage services memory parameters for the node with the ID 7ffa9540-5a20-41d1-b203-e3f349d62565 as follows:

  • the memory guarantee to 8 GB
  • the swap size to 1 GB, and
  • the page cache limits: the minimum to 1 GB, the maximum to 3 GB, and the cache ratio to 0.5.

7.9.5. vinfra memory-policy vstorage-services per-node show

Show per-node memory parameters:

usage: vinfra memory-policy vstorage-services per-node show --node <node>
--node <node>
Node ID or hostname

Example:

# vinfra memory-policy vstorage-services per-node show
--node 7ffa9540-5a20-41d1-b203-e3f349d62565
+-----------+-------------------------+
| Field     | Value                   |
+-----------+-------------------------+
| cache     | maximum: 13194139533312 |
|           | minimum: 8796093022208  |
|           | ratio: 0.7              |
| guarantee |                         |
| swap      | 1099511627776           |
+-----------+-------------------------+

This command lists the storage services memory parameters set for the node with the ID 7ffa9540-5a20-41d1-b203-e3f349d62565.

7.9.6. vinfra memory-policy vstorage-services per-node reset

Reset per-node memory parameters to defaults:

usage: vinfra memory-policy vstorage-services per-node reset [--guarantee] [--swap]
       [--cache] --node <node>
--guarantee
Reset only the guarantee.
--swap
Reset only the swap size.
--cache
Reset only the cache values.
--node <node>
Node ID or hostname

Example:

# vinfra memory-policy vstorage-services per-node reset --cache
--node 7ffa9540-5a20-41d1-b203-e3f349d62565
+-----------+---------------+
| Field     | Value         |
+-----------+---------------+
| cache     |               |
| guarantee |               |
| swap      | 1099511627776 |
+-----------+---------------+

This command resets the manually configured page cache limits to default for the node with the ID 7ffa9540-5a20-41d1-b203-e3f349d62565.