Question

Migrate AWS S3 to Azure Blob


Userlevel 5
Badge

Maybe, we need to change our capactity and archive tier tartget from AWS S3 to Azure. 

What is the best way to migrate AWS S3 to Auzure? Move alle Data from S3 to Azure and do an import  of the repository? Is there a best practise to move 900 TB Data as quick as possible? Is there a way to estimate the cloud out costs?

As I remember I can destroy a AWS Bucket even with object lock enabled. 


4 comments

Userlevel 7
Badge +20

Hi, if you have sufficient rights and your S3 bucket is in governance mode, yes you could delete the data. If it’s compliance mode, I’m not aware of a way to override that, short of engaging AWS support.

 

As for AWS to Azure, it’s not supported to migrate that data. You’d need to download all your backups from your capacity & archive tiers and reupload them to Azure. Your best plan would likely be to create a new SOBR with Azure storage attached and start new retention processing that way.

 

It would be hugely expensive to migrate the data across as you’d need to read every single object (AWS Fee), egress that object (AWS fee), and then write every single object (Azure fee).

 

Your costs to achieve this would vary dramatically based on what tiers of storage you were reading from and writing to

Userlevel 7
Badge +20

Hi, if you have sufficient rights and your S3 bucket is in governance mode, yes you could delete the data. If it’s compliance mode, I’m not aware of a way to override that, short of engaging AWS support.

 

As for AWS to Azure, it’s not supported to migrate that data. You’d need to download all your backups from your capacity & archive tiers and reupload them to Azure. Your best plan would likely be to create a new SOBR with Azure storage attached and start new retention processing that way.

 

It would be hugely expensive to migrate the data across as you’d need to read every single object (AWS Fee), egress that object (AWS fee), and then write every single object (Azure fee).

 

Your costs to achieve this would vary dramatically based on what tiers of storage you were reading from and writing to

I would take this suggestion of starting a new SOBR and a new backup. 900TB is a lot of data to move and will cost you.

Userlevel 7
Badge +6

Agree with these guys.  Migrating 900TB out of anywhere is going to be hugely difficult in both time and money.  I’d also recommend starting a new repo.  If you absolutely need to pull the data from S3, prepare for a long time to download and reupload, lots of fees, and honestly probably a lot of headache.  I personally would just let that data sit, maybe let it migrate off to Glacier or something like that if possible to decrease storage costs, and let it age out there.

Migrating data from AWS S3 to Azure Blob Storage can be done in several ways, depending on your specific requirements and constraints. And you can consider:

  1. Azure Data Box: Azure Data Box is a physical appliance provided by Microsoft that enables offline data transfer. You can use the Data Box to transfer large amounts of data securely by shipping the appliance to your location, copying the data onto it, and then shipping it back to Azure for ingestion. This method can be faster and more efficient for large-scale data transfers.

  2. Azure Data Factory: Azure Data Factory is a cloud-based data integration service that can be used to orchestrate data movement and transformation. It supports copying data from various sources, including AWS S3, to Azure Blob Storage. You can create a data pipeline in Azure Data Factory to transfer your data from AWS S3 to Azure Blob Storage at scale.

  3. Third-party tools: There are third-party tools available that specialize in data migration between different cloud providers. These tools provide features such as data transfer optimization, parallel processing, and monitoring capabilities to help streamline the migration process. Examples of such tools include Gs Richcopy 360 , CloudBerry, Cloudsfer, and MultCloud.

To estimate the cloud cost for your migration, you can use the cloud provider's pricing calculators. Both AWS and Azure provide calculators that allow you to estimate the cost based on factors such as storage size, data transfer, and any additional services you may require. By inputting your data size and expected usage, you can get an estimate of the costs associated with storing your data in Azure Blob Storage.

Comment