Compacting backup archives

Some file systems, such as ext4, XFS, and Btrfs, support a technique called hole punching, which deallocates unused space within files. For example, when data is deleted from an archive on the local storage cluster, the corresponding space is released at the file system level. However, external backup destinations such as NFS shares and public clouds do not support true hole punching; instead, the behavior is only emulated. As a result, unused space accumulates over time.

In public clouds, a 16 MB data chunk can be cleaned up if all of its contents are marked as deleted. However, if even a small portion of the chunk still contains valid data, the entire chunk remains permanently stored, consuming unnecessary space.

To reclaim this unused space, you can manually compact archive files. The compacting tool downloads a backup archive from external storage into memory, skipping regions marked as deleted, and writes the valid data into a new archive on the same external storage. Once completed, the old archive is deleted, and the new one is renamed to match the original.

Downloading and uploading archives can generate significant network traffic. When using hyperscaler public cloud providers, this may result in additional costs for ingress and egress traffic.

Limitations

  • Archives are not compacted if the reclaimable space is less than 32 MB or less than 15% of the archive’s total size.

Prerequisites

  • Before compacting, ensure that your NFS share or cloud bucket has sufficient free space to accommodate a full copy of the compacted archive, which may be large.

To check archive space usage

  1. Identify the archive path relative to the backup storage root (/mnt/vstorage/vols/acronis-backup). For example, the archive path may be /1#12345/1/file.tibx.

  2. Run the vstorage-abgw-ctl compact command with the -i option to view space usage details. Optionally, include the -d0 flag to reduce log output.

    The command will display:

    • Total archive size
    • Size of useful data
    • Overhead (space that can be reclaimed)

    For example:

    # vstorage-abgw-ctl -d0 compact -i /1#12345/1/file.tibx
> source archive: 49.5 MB of useful data out of 198.1 MB total, overhead: 148.7 MB (75.0%)

    In this command output, 198.1 MB is the archive size, 49.5 MB is useful data, and 148.7 MB (75%) is the overhead.

To compact a backup archive

  1. Identify the archive path relative to the backup storage root (/mnt/vstorage/vols/acronis-backup). For example, the archive path may be /1#12345/1/file.tibx.

  2. Run the vstorage-abgw-ctl compact command for the archive. Optionally, include the -d0 flag to reduce log output. For example:

    • On backups with the NFS destination:

      # vstorage-abgw-ctl -d0 compact /1#12345/1/file.tibx
> source archive: 49.5 MB of useful data out of 148.5 MB total, overhead: 99.1 MB (66.7%).
> source archive '/mnt/vstorage/vols/acronis-backup/storage/1#12345/1/file.tibx' deleted
> temporary archive '/mnt/vstorage/vols/acronis-backup/storage/1#12345/1/file.tibx.6dba5fcccb98af473faec499b2349095' renamed to '/mnt/vstorage/vols/acronis-backup/storage/1#12345/1/file.tibx'
> file '/1#12345/1/file.tibx' was compacted

      In this example, the archive size is reduced from 148.5 MB to 49.5 MB, reclaiming 99.1 MB of unused space.

    • On backups with the public cloud destination:

      # vstorage-abgw-ctl -d0 compact /1#12345/2/file.tibx
> source archive: 45.8 MB of useful data out of 92.6 MB total, overhead: 46.8 MB (50.5%).
> source archive '1#12345/2/file.tibx.03ce2a8a97572df64692f7735a4d756c' deleted
> file '/1#12345/2/file.tibx' is now associated with compacted archive '1#12345/2/file.tibx.db8d89387c0bb1496590776f00712b70'

      In this example, the archive size is reduced from 92.6 MB to 45.8 MB, reclaiming 46.8 MB of unused space.