Planning object storage
Use this topic to estimate the storage capacity required for S3-compatible object storage deployments. Object storage requirements depend on usable capacity requirements, redundancy policy, storage media, expected throughput, and projected growth. For predefined deployment architectures, see the S3-only and HCI compute with S3 scenarios. Disk and network requirements are listed in the technical requirements reference.
Object storage prerequisites
- Run production object storage on physical servers.
- Plan additional space for S3 metadata. Object metadata usually requires 0.5-1 percent of the space used by S3 data and can require more when most objects are smaller than 100 KB or users add custom object metadata.
- Since version 4.6, Virtuozzo Infrastructure also keeps object metadata backups. These backups add about 0.5 percent to metadata size, use automatic retention, and follow the same redundancy scheme as the S3 cluster.
The S3 services
S3 object storage includes three service types distributed across the storage nodes:
- Object service (OS): stores object data.
- Name service (NS): stores the object namespace and metadata.
- Gateway (GW): terminates S3 client requests; scale the number of gateways with client throughput.
Capacity planning
Usable capacity is raw capacity minus the erasure-coding overhead and the free-space reserve:
usable = raw × [ M ÷ (M+N) ] × (1 − free_reserve)
raw needed = target_usable ÷ [ M ÷ (M+N) ] ÷ (1 − free_reserve)
- EC efficiency is M ÷ (M+N). For example, 7+2 is about 78 percent, 4+2 is about 67 percent, and 3+2 is about 60 percent. Wider schemes use raw space more efficiently.
- Keep at least 20 percent free per tier for rebuild and write headroom.
Redundancy and tiers
Choose the EC scheme according to the number of nodes. Wide schemes, such as 7+2 on 10 or more nodes, provide low overhead while surviving two failures. Serve different storage classes from separate tiers, such as high-performance NVMe, balanced fast-HDD, and cold SMR tiers.
Sizing S3 services (CPU and RAM)
S3 service reservations are usually small compared with capacity and throughput requirements. Each S3 node runs four S3 gateways, up to 10 object services, and up to 10 name services. The entire S3 cluster cannot host more than 24 object services and 16 name services. The first five S3 nodes run the configd daemon, and all S3 nodes run the hostd daemon.
The reserved RAM size on an S3 node is calculated by using the following formula:
(CONFIGD × 256 MB) + (HOSTD × 256 MB) + (4 × S3GW × 512 MB) + (total_OS × 256 MB + total_NS × 512 MB) ÷ (S3_nodes − nodes_that_can_fail)
The reserved CPU cores on an S3 node are calculated by using the following formula:
(4 × S3GW × 0.5 cores) + (total_OS × 0.1 cores + total_NS × 0.2 cores) ÷ (S3_nodes − nodes_that_can_fail)
Per node, add these values to the system and storage reservations. If EC is used in the storage cluster, also add the storage-client reservation on every node: 5 percent of the node's total RAM plus 2 GiB. In many deployments, a storage/S3 node requires 24-48 GB RAM and 8-16 physical cores, depending on disk count and node role.
Reservation examples
The following examples show how S3 service reservations are added to the system and storage reservations.
| Service | Management nodes |
|---|---|
| System | 4.5 GB RAM, 1 core |
| Storage services | 5 storage disks and 1 metadata service on the system disk: 3 GB RAM, 1.2 cores |
| S3 | 9.7 GB RAM, 4.8 cores |
| Service reservations | 17.2 GB RAM, 7 cores |
| Minimum hardware configuration | 24 GB RAM, 8 cores |
| Recommended hardware configuration | 32 GB RAM, 16 cores |
| Service | Management nodes (nodes 1-3) | Secondary nodes (nodes 4-5) |
|---|---|---|
| System | 4.5 GB RAM, 1 core | 1.5 GB RAM, 1 core |
| Storage services | 10 storage disks, 1 metadata service on the system disk, and 1 cache disk: 6 GB RAM, 2.4 cores | 10 storage disks, 1 metadata service on the system disk, and 1 cache disk: 6 GB RAM, 2.4 cores |
| S3 | 6.2 GB RAM, 3.4 cores | 6.2 GB RAM, 3.4 cores |
| EC storage client | About 3.2 GiB RAM on a 24 GB node | About 3.2 GiB RAM on a 24 GB node |
| Service reservations | 19.9 GB RAM, 6.8 cores | 16.9 GB RAM, 6.8 cores |
| Minimum hardware configuration | 24 GB RAM, 8 cores | 24 GB RAM, 8 cores |
| Recommended hardware configuration | 48 GB RAM, 16 cores | 48 GB RAM, 16 cores |
Performance
S3 throughput is usually limited by the disks and the backend network, not by CPU or RAM. Size the backend NICs according to the expected aggregate object throughput. For capacity at scale, 2 × 25 GbE (LACP) on the backend network is a typical baseline.
Worked example
Target: 1 PB usable, 7+2 erasure coding, 12 nodes.
- Raw capacity: 1 PB ÷ 0.78 ÷ 0.8 gives about 1.6 PB, or about 134 TB per node across 12 nodes. With 18 TB HDDs, plan about eight storage disks per node plus one SSD cache per 4-12 HDDs.
- CPU and RAM: plan 24-48 GB RAM and 8-16 physical cores per node. Include the EC storage-client reservation; on 24-48 GB nodes, it adds about 3.2-4.4 GiB per node. S3 service reservations have limited impact on this estimate.
- Network: use 2 × 25 GbE (LACP) on the backend network to carry the aggregate object throughput.
This example is a planning estimate. Adjust it according to the selected redundancy mode, free-space reserve, disk size, metadata growth, and throughput target.
See also