Configuring the compute cluster for PCI passthrough
To enable PCI passthrough for the compute cluster, you need to create a configuration file in the YAML format and use it to reconfigure the compute cluster.
Below is an example of pci-passthrough.yaml:
- node_id: c3b2321a-7c12-8456-42ce-8005ff937e12 devices: - device_type: sriov device: enp2s0 physical_network: sriovnet num_vfs: 8 - device_type: generic device: 1b36:0100 alias: gpu
In this configuration file:
node_id
is the identifier of a compute node that hosts PCI devicesdevice_type
can besriov
for a network adapter orgeneric
for a graphics carddevice
is the device name for a network adapter or the VID and PID for a graphics cardphysical_network
is an arbitrary name that will be used as an alias for a network adapteralias
is an arbitrary name that will be used as an alias for a graphics card-
num_vfs
is the number of virtual functions to create for a PCI device with thesriov
device typeThe maximum number of virtual functions supported by a PCI device is specified in the /sys/class/net/<device_name>/device/sriov_totalvfs file. For example:
# cat /sys/class/net/enp2s0/device/sriov_totalvfs 63
Now, you can pass this configuration file to the vinfra service compute set
command. For example:
# vinfra service compute set --pci-passthrough-config pci-passthrough.yaml +---------+--------------------------------------+ | Field | Value | +---------+--------------------------------------+ | task_id | 624916c6-617b-4687-b52c-b348e1f4892a | +---------+--------------------------------------+
If the reconfiguration fails with the following error in /var/log/vstorage-ui-backend/ansible.log
:
2021-09-23 16:42:59,796 p=32130 u=vstoradmin | fatal: [32c8461b-92ec-48c3-ae02- 4d12194acd02]: FAILED! => {"changed": true, "cmd": "echo 4 > /sys/class/net/ enp103s0f1/device/sriov_numvfs", "delta": "0:00:00.127417", "end": "2021-09-23 19:42:59.784281", "msg": "non-zero return code", "rc": 1, "start": "2021-09-23 19:42:59.656864", "stderr": "/bin/sh: line 0: echo: write error: Cannot allocate memory", "stderr_lines": ["/bin/sh: line 0: echo: write error: Cannot allocate memory"], "stdout": "", "stdout_lines": []}
In this case, run the the pci-helper.py
script, and reboot the node:
# /usr/libexec/vstorage-ui-agent/bin/pci-helper.py enable-iommu --pci-realloc # reboot
When the node is up again, repeat the vinfra service compute set
command.
After the compute cluster is successfully reconfigured, you can proceed to creating virtual machines.