Enabling S3 cross-region replication
Cross-region replication (CRR) enables automatic, asynchronous copying of objects across S3 buckets in different regions. Buckets that are configured for CRR can be owned by the same user. Objects may be replicated to a single destination bucket or multiple destination buckets.
To enable CRR, you need to add replication configuration to your source bucket. The minimum configuration must provide the destination buckets where you want to replicate objects and a user with the role that enables replicating objects on your behalf.
Limitations
- CRR only supports copying new S3 objects after it is enabled.
Prerequisites
- S3 clusters are created, as described in Creating the S3 cluster.
- To be able to use the AWS command-line tool, install and configure it first, as described in the official documentation Install/Update and Setup.
To set up CRR replication
-
Create two or more S3 buckets, one bucket in the source S3 cluster and one or multiple buckets in the destination S3 cluster. You can also use buckets that already exist in your S3 clusters. For example, to create the
sourceanddestinationbuckets, use:# ostor-s3-cmd create-bucket -e http://s3.ostor_source -k f6995711d730d7dc3T0T -s 3tNgp4aGJYjtDG8UFdzHoFR8HKMyV5qD0DWiGMBP -b source # ostor-s3-cmd create-bucket -e http://s3.ostor_destination -k c460186c7b91a79f -s z4Fw8p8y2jTdjusTme2uBaLDe6p69F5yUPFuHHz8 -b destination
Where:
-eis the URL of the S3 cluster where to create a bucket.-kand-sare the access and secret keys of the S3 user.-bspecifies a bucket name.
-
Enable versioning for these buckets. For example:
# ostor-s3-cmd enable-versioning -e http://s3.ostor_source -k f6995711d730d7dc3T0T -s 3tNgp4aGJYjtDG8UFdzHoFR8HKMyV5qD0DWiGMBP -b source # ostor-s3-cmd enable-versioning -e http://s3.ostor_destination -k c460186c7b91a79f -s z4Fw8p8y2jTdjusTme2uBaLDe6p69F5yUPFuHHz8 -b destination
-
Create a replication configuration file. For example, the replication.conf file may look as follows:
{ "Role": "arn:aws:iam::f6995711d730d7dc:role/service-role/s3crr_role", "Rules": [ { "ID": "rep-rule", "Status": "Enabled", "Priority": 1, "Filter": {}, "Destination": { "Bucket": "arn:aws:s3:::destination" }, "DeleteMarkerReplication": { "Status": "Disabled" } } ] }Where:
Roleis the ID of the user that is used to replicate objects on your behalf. In the example above, it isf6995711d730d7dc.Rulespecifies which objects to replicate and where to store the replicas.IDis a unique identifier for the rule. The value must be up to 255 characters long.Statusspecifies whether the rule is enabled. Valid values areEnabledorDisabled.Priorityindicates which rule has precedence whenever two or more replication rules conflict.Bucketis the name of the bucket where you want to store the results.Destinationis a container for information about the replication destination and its configuration.DeleteMarkerReplicationspecifies whether to replicate delete markers. If theFilterelement is specified, you must also include theDeleteMarkerReplicationelement.
-
Configure the source bucket for CRR by specifying the replication configuration file. For example:
# ostor-s3-cmd put-bucket-replication-config -e http://s3.ostor_source -k f6995711d730d7dc3T0T -s 3tNgp4aGJYjtDG8UFdzHoFR8HKMyV5qD0DWiGMBP \ -b source -c replication.conf -D http://s3.ostor_destination -K c460186c7b91a79f -S z4Fw8p8y2jTdjusTme2uBaLDe6p69F5yUPFuHHz8
Where:
-eis the URL of the source S3 cluster.-kand-sare the access and secret keys of the user that is used to replicate objects on your behalf. In the example above, it isf6995711d730d7dc.-bis the name of the source bucket.-cspecifies the replication configuration file.-Dis the URL of the destination S3 cluster.-Kand-Sare the access and secret keys of the user that has access to the destination buckets.