6.4. Enabling RDMA¶
This feature is experimental and not recommended for production until version 4.0.
Virtuozzo Hybrid Infrastructure supports remote direct memory access (RDMA) over Converged Ethernet (RoCE), Internet Wide-area RDMA Protocol (iWARP), or InfiniBand (IB) for the storage backend network. The RDMA technology allows servers in this network to exchange data in main memory without involving their processors, cache or operating systems, thus freeing up resources and improving throughput and performance.
Your RDMA network infrastructure must be ready before you install Virtuozzo Hybrid Infrastructure.
In the current version of Virtuozzo Hybrid Infrastructure, you can only enable (or disable) RDMA before creating the storage cluster.
By default, RDMA is disabled. Before enabling it, make sure that each network adapter connected to a network with the Storage traffic type supports RDMA.
To enable or disable RDMA, use the switcher on the SETTINGS > Advanced settings > RDMA tab. Changing this option may temporarily affect cluster availability.
6.4.1. Configuring InfiniBand Devices¶
As the admin panel only shows IP states and does not show InfiniBand connection states, it may report plugged in but yet unconfigured IB devices as UNPLUGGED. The status will change to OK once you assign an IP address to such a device.
If you have an InfiniBand infrastructure, do the following before enabling RDMA:
Assign the traffic type Storage to an empty network (without any other traffic types) on the INFRASTRUCTURE > Networks screen. Create a new network if needed by clicking Edit > Create network.
Configure each IB device on registered nodes:
Open INFRASTRUCTURE > Nodes > <node> > Network, and select the device.
Click Configure. In the pane that opens:
- Assign an IP address (Manually will be selected by default).
- Specify the gateway.
- Check the Connected mode box.
- Set MTU to 65520.
Click Assign network. In the pane that opens, select the network with just the Storage traffic type and click DONE.
6.4.2. Configuring RoCE and iWARP Devices¶
If you have a RoCE or iWARP infrastructure, do the following for each network device in it before enabling RDMA:
- Open INFRASTRUCTURE > Nodes > <node> > Network, and select the device.
- Click Configure. In the pane that opens, assign an IP address if the device does not have one yet and specify the gateway. Click DONE.
- Click Assign network. In the pane that opens, select the network with the Storage traffic type (and possibly other traffic types like Internal management, OSTOR private, and ABGW private) and click DONE.