Deploying the storage cluster

Create the storage cluster on one (the first) node, then populate it with more nodes.

Limitations

You can assign a role to a disk only if its size is greater than 1 GiB.
You can assign an additional role to a system disk only if its size is at least 100 GiB.
You can use shingled magnetic recording (SMR) HDDs only with the Storage role and only if the node has an SSD disk with the Cache role.
You cannot use SMR and standard disks in the same tier.
You cannot assign roles to system and non-system disks at a time.

Prerequisites

A clear understanding of the storage cluster architecture and disk roles, which are explained in About the storage cluster.
A clear understanding of the concept Storage tiers.
Your infrastructure networks are set up, as described in Setting up networks.
The node network interfaces are configured by following the instructions in Configuring node network interfaces.
If supported, RDMA is enabled, as described in Enabling RDMA.
External DNS servers are added automatically during the installation or manually, as described in Adding external DNS servers.
All of the nodes are shown in the admin panel on the Infrastructure > Nodes screen with the Unassigned status.

To create the storage cluster on the first node

Admin panel

Open the Infrastructure > Nodes screen, and then click Create storage cluster.
To configure the disk roles or node location, click the cogwheel icon.
- To configure disks
  1. Select Disks.
  2. Select the required disk, and then click Configure.
  3. In the Choose role window, select a role:
    
    [Only for SSD drives] To store write cache
    
    Select the Cache role.
    
    Select a storage tier that you want to cache.
    
    For storage disks to use cache, the Cache role must be assigned before the Storage role.
    
    To store data
    
    Select the Storage role.
    
    Select a storage tier where to store your data. To make better use of data redundancy, do not assign all of the disks on a node to the same tier. Instead, make sure that each tier is evenly distributed across the cluster with only one disk per node assigned to it.
    
    Enable data caching and checksumming:
    
    Enable SSD caching and checksumming. Available and recommended only for nodes with SSDs.
    
    Enable checksumming (default). Recommended for nodes with HDDs as it provides better reliability.
    
    Disable checksumming. Not recommended for production. For an evaluation or testing environment, you can disable checksumming for nodes with HDDs, to provide better performance.
    
    To store cluster metadata
    
    Select the Metadata role.
    
    It is recommended to have only one metadata service per node and maximum five metadata services in a cluster.
    
    [Only for SSD drives] To store both metadata and write cache
    
    Select the Metadata+Cache role.
    
    Select a storage tier that you want to cache.
  4. Repeat the previous steps for every disk you want to be used in the storage cluster, and click Done.
  5. On the Configuration summary screen, check the number of disks per each configuration category, and then click Proceed.
- To configure node location
  By default, the nodes are added to the Default rack in the Default row in the Default room.
  1. Select Location.
  2. To create a room, click Create room in the Move nodes window. Specify the room name, and then click Create.
  3. To add a new row, click the created room. Click Create row, enter a name for it, and then click Create.
  4. To add a new rack, click the created row. Click Create rack, enter a name for it, and then click Create.
  5. Click the created rack, and then click Move.
Enter a name for the cluster. It may only contain Latin letters (a-z, A-Z), numbers (0-9), and hyphens (“-“).
Enable encryption, if required.
Click Create.

You can monitor cluster creation on the Infrastructure > Nodes screen. The creation might take some time, depending on the number of disks to be configured. Once the configuration is complete, the cluster is created.

Command-line interface

Use the following command:

vinfra cluster create [--disk <disk>:<role>[:<key=value,…>]] [--tier-encryption {0,1,2,3}]
                      --node <node> <cluster-name>

--disk <disk>:<role> [:<key=value,…>]

Disk configuration in the format:

<disk>: disk device ID or name
<role>: disk role (cs, mds, journal, mds-journal, mds-system, cs-system, system)
comma-separated key=value pairs with keys (optional):
- tier: disk tier (0, 1, 2 or 3)
- journal-tier: journal (cache) disk tier (0, 1, 2 or 3)
- journal-type: journal (cache) disk type (no_cache, inner_cache or external_cache)
- journal-disk: journal (cache) disk ID or device name
- bind-address: bind IP address for the metadata service

Example: sda:cs:tier=0,journal-type=inner_cache.
This option can be used multiple times.

--tier-encryption {0,1,2,3}

Enable encryption for storage cluster tiers. Encryption is disabled by default. This option can be used multiple times.

--node <node>

Node ID or hostname

<cluster-name>

Storage cluster name

For example, to create the storage cluster stor1 on the node node001, run:

# vinfra cluster create stor1 --node node001

As disk roles are not explicitly specified, they are assigned automatically: mds-system to the system disk, and cs to all other disks.

You can view the storage cluster details in the vinfra cluster show output:

# vinfra cluster show
+-------+--------------------------------------------+
| Field | Value                                      |
+-------+--------------------------------------------+
| id    | 1                                          |
| name  | stor1                                      |
| nodes | - host: node001.vstoragedomain             |
|       |   id: f59dabdb-bd1c-4944-8af2-26b8fe9ff8d4 |
|       |   is_installing: false                     |
|       |   is_releasing: false                      |
+-------+--------------------------------------------+

To add nodes to the cluster