RELATED: How to Transfer an S3 Bucket to Google Cloud Platform Storage Setting Up rclone If you’re moving to a platform that supports this, you’ll likely want to use their service instead.
Some cloud services, like Google Cloud Platform, have services of their own that can handle the transfer. If that is a problem, there are other tools available to seamlessly transfer, including commercial tools like NetApp Cloud Sync that can sync multiple buckets together. You will likely need a bit of downtime to ensure the buckets are synced up before finally switching over. Luckily, you can perform the large initial transfer while the bucket is still actively being written to. If you have millions of files, or multiple terabytes, you should be prepared for hours of transfer times. Number of files is also an issue, as rclone adds overhead for each transfer. rclone can also handle file updates, which can be useful if the source bucket is being written to as it’s transferring.Īs far as transfer times go, it will likely take a while depending on the size of the bucket. To handle the transfer, rclone will read from the source bucket, find all the files that need to be transferred, and handle cloning them into the destination bucket. For example, you could transfer from S3 to Digital Ocean’s compatible Spaces service, or transfer from an S3 bucket in one account to another account. If both services you’re transerring to and from are S3 compatible, you can simply use a utility like rclone, configured to access each service, to transfer all the items over. If you want to move off AWS, transferring an S3 bucket is easy to do. AWS’s Simple Storage Service (S3) is great for storing large amounts of objects, but it’s also an API that’s compatible with many other competiting services.