Preparing nodes for PCI passthrough

For SR-IOV virtualization, check that the network adapter that you want to pass through can be virtualized:

List all network adapters on a node and obtain their VID and PID:

# lspci -nn | grep Ethernet
00:03.0 Ethernet controller [0200]: Mellanox Technologies MT27800 Family [ConnectX-5] [15b3:1017]
00:04.0 Ethernet controller [0200]: Mellanox Technologies MT27800 Family [ConnectX-5] [15b3:1017]

[15b3:1017] is the VID and PID of the network adapter.

Check that the chosen network adapter supports SR-IOV by using its VID and PID:

# lspci -vv -d 15b3:1017 | grep SR-IOV
Capabilities: [180 v1] Single Root I/O Virtualization (SR-IOV)
Capabilities: [180 v1] Single Root I/O Virtualization (SR-IOV)

For GPU passthrough, detach the GPU card that you want to attach to VMs from the host.
1. List all graphics cards on a node and obtain their VID and PID:
```
# lspci -nn | grep VGA
03:00.0 VGA compatible controller [0302]: NVIDIA Corporation TU104GL [Tesla T4] [10de:1eb8] (rev a1)
03:00.0 VGA compatible controller [0302]: NVIDIA Corporation TU104GL [Tesla T4] [10de:1eb8] (rev a1)
```
  [10de:1eb8] is the VID and PID of the graphics card.
2. Run the pci-helper.py script to detach the chosen GPU specifying its VID and PID. For an NVIDIA graphics card, additionally blacklist the Nouveau driver. For example:
```
# /usr/libexec/vstorage-ui-agent/bin/pci-helper.py detach 10de:1eb8 --blacklist-nouveau
```
Enable IOMMU on the node by running the pci-helper.py script:
```
# /usr/libexec/vstorage-ui-agent/bin/pci-helper.py enable-iommu
```
The script works for both Intel and AMD processors.
Reboot the node to apply the changes.

You can check that IOMMU is successfully enabled in the dmesg output:

# dmesg | grep -e DMAR -e IOMMU
[    0.000000] DMAR: IOMMU enabled

For Mellanox network adapters, you need to additionally enable SR-IOV in firmware. Do the following:

Download Mellanox Firmware Tools (MFT) from the official website and extract the archive on the node. For example:

# wget https://www.mellanox.com/downloads/MFT/mft-4.17.0-106-x86_64-rpm.tgz
# tar -xvzf mft-4.17.0-106-x86_64-rpm.tgz

Install the package, and then start Mellanox Software Tools (MST):

# yum install rpm-build
# . mft-4.17.0-106-x86_64-rpm/install.sh
# mst start

Determine the MST device path:
```
# mst status
```

Query the current configuration:

# mlxconfig -d /dev/mst/mt4119_pciconf0 q
...
Configurations:
...
         NUM_OF_VFS            4               # Number of activated VFs
         SRIOV_EN              True(1)         # SR-IOV is enabled
...

Set the desired values, if necessary. For example, to increase the number of virtual functions to 8, run:
```
# mlxconfig -d /dev/mst/mt4119_pciconf0 set SRIOV_EN=1 NUM_OF_VFS=8
```
Reboot the node to apply the changes.