Enabling S3 cross-region replication

Cross-region replication (CRR) enables automatic, asynchronous copying of objects across S3 buckets in different regions. Buckets that are configured for CRR can be owned by the same user. Objects may be replicated to a single destination bucket or multiple destination buckets.

To enable CRR, you need to add replication configuration to your source bucket. The minimum configuration must provide the destination buckets where you want to replicate objects and a user with the role that enables replicating objects on your behalf.

Limitations

  • CRR only supports copying new S3 objects after it is enabled.

Prerequisites

  • S3 clusters are created, as described in Creating the S3 cluster.
  • To be able to use the AWS command-line tool, install and configure it first, as described in the official documentation Install/Update and Setup.

To set up CRR replication

  1. Create two or more S3 buckets, one source bucket and one or multiple destination buckets. You can also use buckets that already exist in the S3 cluster. For example, to create the source and destination buckets, use:

    # aws s3api create-bucket --bucket source --endpoint-url http://s3.ostor --profile ostor
    # aws s3api create-bucket --bucket destination --endpoint-url http://s3.ostor --profile ostor
  2. Enable versioning for these buckets. For example:

    # aws s3api put-bucket-versioning --bucket source --endpoint-url http://s3.ostor --profile ostor \
    --versioning-configuration 'Status=Enabled'
    # aws s3api put-bucket-versioning --bucket destination --endpoint-url http://s3.ostor --profile ostor \
    --versioning-configuration 'Status=Enabled'
  3. Create a replication configuration file. For example, the replication.conf file may look as follows:

    <?xml version="1.0"?>
    <ReplicationConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
      <Rule>
        <Status>Enabled</Status>
        <Filter/>
        <DeleteMarkerReplication>
          <Status>Disabled</Status>
        </DeleteMarkerReplication>
        <Destination>
          <Bucket>arn:aws:s3:::destination</Bucket>
        </Destination>
        <Priority>1</Priority>
        <ID>rep-rule</ID>
      </Rule>
      <Role>arn:aws:iam::ccb13f7bc586ace0:role/service-role/s3crr_role</Role>
    </ReplicationConfiguration>

    where:

    Rule
    Specifies which objects to replicate and where to store the replicas.
    DeleteMarkerReplication
    Specifies whether to replicate delete markers. If the Filter element is specified, you must also include the DeleteMarkerReplication element.
    Destination
    A container for information about the replication destination and its configuration.
    Priority
    Indicates which rule has precedence whenever two or more replication rules conflict.
    Bucket
    The name of the bucket where you want to store the results.
    Status
    Specifies whether the rule is enabled. Valid values are Enabled or Disabled.
    ID
    A unique identifier for the rule. The value must be up to 255 characters long.
    Role
    The ID of the user that is used to replicate objects on your behalf. In the example above, it is ccb13f7bc586ace0.
  4. Configure the source bucket for CRR by specifying the replication configuration file. For example:

    # aws s3api put-bucket-replication --bucket source --endpoint-url http://s3.ostor --profile ostor \
    --replication-configuration file://replication.conf