Cloud Failures, Clouds on Fire, Clouds… Underwater? A Reminder on the Importance of Data Protection in the Cloud

So...the interesting thing for me is that a GCP “region” is a single point of failure. I know that AWS definition of “region” has at least three availability zones.

And I will echo and paraphrase @MicoolPaul in saying “Nobody expects the Spanish Inquisition!”

The GCP example shows that you shouldn't store your backups in the same data center or region. In the worst case you could lose both your production and your backup data.

@HangTen416 makes some good points here.

Firstly the architecture differences between AWS/Azure/GCP were described to me in the following sentences:

Amazon runs on AWS, Microsoft runs on Azure, GCP runs on Google.

And I don’t say that as an insult to GCP, but I think it provides valuable insight to the way design decisions would be made with their products in the future.

Furthermore, and this sounds ridiculous to say, you have to trust, or truly understand, your hosting/cloud provider’s architecture to get the best out of it.

Looking at OVH’s 2021 fire in Strasbourg, there were two technical design decisions that I feel are worth shining a light on:

• Backups were stored within the same datacentre buildings, despite a contractual insinuation that they would be stored as multiple separate copies. I won’t provide too much detail here as this has already been taken to court and a ruling against OVH was made judged on the incompatibility between their delivered backup service providing local backups, and that being unable to meet the contracted terms of service offered to the customers. Link for details: https://www.datacenterdynamics.com/en/news/ovhcloud-ordered-to-pay-250k-to-two-customers-who-lost-data-in-strasbourg-data-center-fire/
• The information on this is quite murky, but from my understanding OVH had four physical buildings in Strasbourg, SBG-1 through to 4. But OVH offered an SBG-5 as well, but this was a public cloud offering that was only a virtual datacentre, running across multiple racks within SBG 1 to 4. This meant that when people were trying to get reassurance from their data not being in the impacted SBG-1 or SBG-2 regions, they were then unexpectedly losing data they thought was unaffected. This is based on information I found on Twitter during the time of the event, but it’s harder to find details now that OVH have launched a physical SBG-5 to replace the destroyed SBG-2. The reason I talk about this though is if you expected to have multiple local copies within a region such as SBG-2 and SBG-5, you’d be frustrated to find out they were actually just rack neighbours and both copies were gone. Extra bad luck points if your third copy was in an impacted SBG-1 rack!