Sync AWS S3 Bucket to Backblaze

Back in the day, we stored backups on-site, using everything from servers to removable hard drives. These media, however, were prone to failures that could lead to data loss and its costly consequences. Fortunately, cloud backups have now become the norm. They allow you to back up your data quickly and easily without needing any maintenance or updates (and with lower failure risks).

But what happens when your cloud provider does suffer a failure? This is where data replication backups come in.

By simply storing the same data across different storage providers, you’ll ensure data accessibility and availability at all times. In fact, data replication backups should be an essential component of your overall data recovery strategy! Replicating S3 buckets to Backblaze provides you with data redundancy and assurance.

So today, we’ll show you how to sync your AWS S3 bucket to Backblaze using SimpleBackups, so you always have a reliable backup should anything go wrong.

Prepare Your AWS S3 Bucket for Sync

Let’s start by setting up an AWS S3 bucket. To do this, log into your AWS console, click on Storage in the Services menu, and then on S3. Click on Create bucket on the page that opens.

On the page that opens:

• Give your bucket a name.
• Select the AWS region you’d like to use to host your bucket.
• Provide configuration information about the bucket, including determining the level of public access you require for the bucket.
• Click Create bucket when you’re satisfied with all the other settings.

To allow access to SimpleBackups to run the backup, you’ll also need to set up a user with the right permissions. Here:

• Go to your AWS console, click on the Services menu, and then on Security, Identity, and Compliance. Then click on IAM.

• Click on Users in the menu on your IAM dashboard. 

• Click on Create User and provide a name for the user you want to create. 

• Select the Attach policies directly option, search for S3, and you may select the AmazonS3FullAccess option (this is discouraged for production accounts, and is not recommended actually, so please ensure you limit access as shown at the end of this article for the minimum permissions needed).

• Review all your settings on the final page and click Create user if you’re satisfied. 

When the user is created, you’ll also need to generate the credentials you’ll use to give SimpleBackups access to your AWS S3 bucket. To do this, click on Users on your IAM dashboard and click on the user in the list of users. 

Click on the Security credentials tab, scroll down to Access keys, and click on Create access key. On the next page, click Next, and then provide a description tag and click Create access key. On the confirmation screen, copy and paste the access so you can use it later to connect your bucket to SimpleBackups. 

Prepare Backblaze Bucket for Sync

Next, you’ll need to configure your Backblaze B2 bucket. To do this, log into your Backblaze account and choose Buckets under B2 Cloud Storage in the left menu. Click on Create a Bucket. 

Provide the information about the bucket you want to create for your backups and click Create a Bucket. 

To give SimpleBackups access to your Backblaze bucket, you’ll also need to generate the credentials you’ll use. To do this, go to App Keys in the left menu and click Add a New Application Key. 

In the dialog box that opens, provide a name for the key, choose the bucket you’d like to give SimpleBackups access to, and select Read and Write access. 

Once done, click Create New Key. You’ll then get the confirmation that your key was generated. Save these credentials because you’ll use them later. 

Replicate your S3 bucket data to Backblaze

Now that we’ve set up both the AWS and Backblaze buckets, we can now back up the S3 bucket to Backblaze. 

Here, the first step is to log into your SimpleBackups account and click on Storage Sync in the main menu. On the page that opens, click on Create Storage Sync. 

You can also click on Storage Replication Backup further down on the page. Either way, you’ll go to the Create Storage Sync page. 

When setting up your backup, your first choice will be whether you’d like to use your own server or use SimpleBackups’ infrastructure to run the backups. Here, you’ll choose either the Own Server or, as is the case in this example, the Serverless option. 

Next, you’ll decide when you’d like your backup to run. You can choose between Daily, Weekly, Monthly, Custom, or On Demand based on your unique requirements. In this example, choose Daily from the drop-down list. 

During the next step, you’ll choose both your source and destination storage providers. Because you’ve only set up your storage buckets during the earlier steps, you’ll need to connect them first. To connect your AWS S3 bucket as Storage Source:

• Click on Connect storage.
• Choose Amazon S3 as the provider in the dialog box that opens. 
• Enter the bucket’s information and the credentials you obtained earlier. 
• Once done, click on Validate and then on Save Storage.

You’ll follow the same process to connect your Backblaze bucket as Destination Storage. So, you’ll:

• Click on Connect storage.
• Choose Backblaze B2 from the drop-down list in the dialog box. 
• Enter the bucket’s information and credentials.
• Once done, click on Validate and then Save Storage. 

Once you’ve connected both buckets to SimpleBackups, click on Validate to validate the connection. 

Finally, you’ll give your backup a name. In this example, name it S3toB2Backup. 

Once done, click Create Storage Sync to finalize your backup. 

Securing S3 Sync via Policies

Limit S3 access/scope via IAM policy, and limiting S3 Permissions

To create an S3 storage user for SimpleBackups, the following minimum permissions are required to be available on the backup bucket of your choice:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:AbortMultipartUpload",
                "s3:CompleteMultipartUpload",
                "s3:CreateMultipartUpload",
                "s3:ListMultipartUploadParts",
                "s3:ListBucketMultipartUploads"
            ],
            "Resource": [
                "arn:aws:s3:::MY_BACKUPS_BUCKET/*",
                "arn:aws:s3:::MY_BACKUPS_BUCKET"
            ]
        }
    ]
}

The policy above can be used to only allow SimpleBackups to work on the backups bucket of your choice.

Don't forget to change MY_BACKUPS_BUCKET to the actual buckets you create for backups.

Note #1:

The following permissions are needed for cleaning up uncompleted uploads and save storage space.

• s3:AbortMultipartUpload
• s3:CompleteMultipartUpload
• s3:CreateMultipartUpload
• s3:ListMultipartUploadParts
• s3:ListBucketMultipartUploads

Note #2:

You may fully remove write access and all permissions that allow deleting objects from the bucket if you don't want SimpleBackups to delete any files from the bucket. SimpleBackups allows you to connect read-only and object-locked (immutable) buckets.

Wrapping up

If you’d like your data to be available at all times, you should implement data replication backups as part of your data recovery strategy. You’ll have peace of mind knowing that your data will be safe, no matter what happens to your cloud storage provider! 

Hopefully, this post helped illustrate how you can set up and run backups between Amazon AWS S3 and Backblaze quickly and easily. The only thing left to do now is to get started with SimpleBackups!