Network requirements

These requirements describe the network layout, bandwidth, MTU, traffic separation, and external-access requirements for Virtuozzo Infrastructure.

Backend and frontend networks

Plan at least two traffic domains:

• Backend or internal network: carries storage, internal management, S3 private, Backup Gateway private, and compute virtual-network traffic. At minimum, use 10 Gbit/s bandwidth. For production, use bonded 10+ GbE links or faster.
• Frontend or public network: carries admin panel, self-service panel, APIs, VM public, S3 public, Backup Gateway public, iSCSI, NFS, SMB, SSH, SNMP, DNS, and NTP traffic, depending on enabled services. For external traffic, you can start with 1 GbE links, but they can limit throughput on modern workloads.

The reference production layout uses two bonded networks with LACP: two ports for the backend network and two ports for the frontend network.

Traffic types are normally separated with VLANs or separate interfaces. For production, use separate networks for internal and public traffic. Expose public services through NAT and port forwarding at the edge, and avoid exposing the admin panel, Compute API, and SSH directly to the Internet.

General network requirements

• All network interfaces on a node must be assigned IP addresses from different subnets, as having multiple interfaces in the same subnet can cause various issues. A network interface can be a VLAN-tagged logical interface, an untagged bond, or an Ethernet link.
• Public network interfaces on cluster nodes can be assigned IP addresses from different subnets. Depending on your network infrastructure, you can interconnect these nodes either by placing them within a single L2 broadcast domain (VLAN) or by routing traffic between subnets through an L3 router.
• The network for internal traffic can be non-routable, with minimum 10 Gbit/s bandwidth. Node interfaces connected to this network must be assigned IP addresses from the same subnet.
• Nodes are added to clusters by their IP addresses, not FQDNs. Changing the IP address of a node in the cluster will remove that node from the cluster. Use static IP addresses for infrastructure interfaces. If you plan to use DHCP in a cluster, make sure that IP addresses are bound to the MAC addresses of the nodes’ network interfaces.
• Configure a single gateway per node, on the public network. Multiple gateways can break routing.
• Each node must have Internet access, directly or through a proxy, so that updates can be installed.
• Network time synchronization is required for correct statistics, compute initialization, live migration, and token validation. It is enabled by default via the chronyd service. If you want to use ntpdate or ntpd, stop and disable chronyd first.
• Use an external firewall for untrusted public networks, such as the Internet.

• If you use NVIDIA Mellanox network adapters with AMD EPYC Rome CPUs, ensure that SR-IOV is properly enabled. Otherwise, data loss and performance degradation may occur.

  To enable SR-IOV

  1. Enable SR-IOV in BIOS.

  2. Enable IOMMU on the node:

     1. In the /etc/default/grub file, locate the GRUB_CMDLINE_LINUX line, and then add the iommu=pt kernel parameter. The resulting line may look as follows:

        GRUB_CMDLINE_LINUX="crashkernel=auto tcache.enabled=0 quiet iommu=pt"

     2. Regenerate the GRUB configuration file by running:

        # grub2-mkconfig -o /boot/grub2/grub.cfg --update-bls-cmdline

        The default location is different on a UEFI-based system.

Bandwidth and MTU sizing

• For balanced storage performance, provide about 1 Gbit/s of backend bandwidth per two HDDs on a node, rounded up.
• For maximum sequential I/O performance, provide 1 Gbit/s per HDD, or one 10+ Gbit/s link per node.
• For maximum overall performance, use 25 or 40 Gbit/s network adapters. Using 10 Gbit/s adapters is possible but not recommended for high-performance production clusters.
• Do not configure 1 Gbit/s adapters to use jumbo frames. Configure 10+ Gbit/s adapters to use MTU 9000, or the maximum MTU supported end to end by your switches, routers, node NICs, bonds, and VLANs.

Storage network interface	Node max. I/O	VM max. I/O (replication)	VM max. I/O (erasure coding)
1 Gbps	100 MB/s	100 MB/s	70 MB/s
2 × 1 Gbps	About 175 MB/s	100 MB/s	About 130 MB/s
3 × 1 Gbps	About 250 MB/s	100 MB/s	About 180 MB/s
10 Gbps	1 GB/s	1 GB/s	700 MB/s
2 × 10 Gbps	1.75 GB/s	1 GB/s	1.3 GB/s
25 Gbps	About 2.5 GB/s	About 1 GB/s	About 1.8 GB/s
2 × 25 Gbps	About 4.5 GB/s	About 1 GB/s	About 3.2 GB/s

The 25 Gbit/s figures are estimates extrapolated from 10 Gbit/s measurements.

Network ports

Ports that will be opened on cluster nodes depend on services that will run on the node and traffic types associated with them. Before enabling a specific service on a cluster node, you need to assign the respective traffic type to a network this node is connected to. Assigning a traffic type to a network configures a firewall on nodes connected to this network, opens specific ports on node network interfaces, and sets the necessary iptables rules.

The table below lists all the required ports and services associated with them:

Service	Traffic type	Port	Traffic direction	Description
Web control panel	Admin panel1 Ports for this traffic type must only be open on management nodes.	TCP 8888	Inbound	External access to the admin panel.
	Self-service panel	TCP 8800	Inbound	External access to the self-service panel.
Management	Internal management	all ports for a subnet	Inbound/outbound	Internal cluster management and transfers of node monitoring data to the admin panel.
Metadata service	Storage	all ports for a subnet	Inbound/outbound	Internal communication between MDS services, as well as with chunks services and clients.
Chunk service		all ports for a subnet	Inbound/outbound	Internal communication with MDS services and clients.
Client		all ports for a subnet	Inbound/outbound	Internal communication with MDS and chunk services.
Backup Gateway	Backup (ABGW) public	TCP 40440, 44445	Inbound	External data exchange with Acronis Cyber Protect agents and Acronis Cyber Protect Cloud.
	Backup (ABGW) private	all ports for a subnet	Inbound/outbound	Internal management of and data exchange between multiple backup storage services.
	—	TCP 8443	Outbound	Data control for Acronis Cyber Protect agents and Management server
	—	TCP 9877	Outbound	Registration with Acronis Cyber Protect Management server in on-premises installations
iSCSI	iSCSI	TCP 3260	Inbound	External data exchange with the iSCSI access point.
S3	S3 public	TCP 80, 443	Inbound	External data exchange with the S3 access point.
	OSTOR private	all ports for a subnet	Inbound/outbound	Internal data exchange between multiple S3 services.
NFS	NFS	TCP/UDP 111, 892, 2049	Inbound	External data exchange with the NFS access point.
	OSTOR private	all ports for a subnet	Inbound/outbound	Internal data exchange between multiple NFS services.
File storage	SMB	TCP 139, 445	Inbound	External data exchange with the SMB access point.
Compute	Compute API2 Ports for this traffic type must only be open on management nodes.			External access to standard OpenStack API endpoints:
		TCP 5000	Inbound	Identity API v3
		TCP 6080	Inbound	noVNC Websocket Proxy
		TCP 8004	Inbound	Orchestration Service API v1
		TCP 8041	Inbound	Gnocchi API (billing metering service)
		TCP 8774	Inbound	Compute API
		TCP 8776	Inbound	Block Storage API v3
		TCP 8780	Inbound	Placement API
		TCP 9292	Inbound	Image Service API v2
		TCP 9313	Inbound	Key Manager API v1
		TCP 9513	Inbound	Container Infrastructure Management API (Kubernetes service)
		TCP 9696	Inbound	Networking API v2
		TCP 9888	Inbound	Octavia API v2 (load balancer service)
	VM public	L2 layer	Inbound/outbound	External data exchange between VMs and public networks.
	VM private	UDP 4789	Inbound/outbound	Network traffic between VMs in compute virtual networks.
		TCP 15900–16900	Inbound/outbound	VNC console traffic.
	VM backups	TCP 49300–65535	Inbound/outbound3 Outbound traffic is only used for the Acronis Disaster Recovery (DR) hybrid deployment.	External access to NBD endpoints.
	—	UDP 500, 4500	Outbound	VPN as a Service
SSH	SSH	TCP 22	Inbound	Remote access to nodes via SSH.
SNMP	SNMP4 Ports for this traffic type must only be open on management nodes.	UDP 161	Inbound	External access to storage cluster monitoring statistics via the SNMP protocol.
DNS	—	TCP/UDP 53	Outbound	DNS name resolution.
NTP	—	UDP 123	Outbound	Time syncronization.

VLAN trunking

VLAN trunking allows a single physical interface (trunk) to carry traffic for multiple VLANs simultaneously. In Virtuozzo Infrastructure, VLAN trunking is provided by either the Linux kernel subsystem or Open vSwitch (OVS). Understanding how they differ is essential for network design.

Linux VLANs

Linux VLANs are created directly on physical or bonded network interfaces (for example, eth0 or bond0). Each VLAN is represented as a subinterface, which allows separate configuration per VLAN. For Linux VLANs, the kernel handles VLAN tag insertion and removal.

The parent interface functions as a trunk, carrying all VLAN-tagged traffic for its subinterfaces.

Example configuration:

• eth0 is the parent Ethernet interface
• eth0.100 and eth0.200 are Linux VLANs

Linux VLANs are configured in the infrastructure and can be used for the storage, internal management, and VXLAN traffic.

OVS VLANs

OVS VLANs are created on bridge interfaces (for example, br-eth0 or br-bond0), which function as software switches. OVS handles VLAN tagging and forwarding internally, maintaining a switching table. OVS bridges operate as trunk interfaces, allowing multiple VLANs to pass through a single uplink.

Example configuration:

• eth0 is the Ethernet interface
• br-eth0 is the bridge interface
• br-eth0.1 and br-eth0.2 are OVS VLANs

OVS VLANs are configured in the compute cluster and used for VM public traffic.

Mixed VLAN deployments

Linux VLANs and OVS VLANs can coexist, but mixing them requires careful planning to avoid conflicts.

Example scenario:

1. Linux VLANs are created on the eth0 Ethernet interface: eth0.100, eth0.200.
2. Later, the compute cluster is created using the same physical interface. This creates an OVS bridge br-eth0, turning it into an OVS trunk interface.
3. VLANs created within the compute cluster become OVS VLANs: br-eth0.1 and br-eth0.2.