Skip to main content

The sum of all fears: Ransomware attacking S3 buckets


Michael Melter
Forum|alt.badge.img+12

Most S3 offerings allow to leverage customer provided keys to encrypt data uploaded there (SSE-C).

By calling the x-amz-server-side-encryption-customer-algorithm an encryption process inside your buckets can be initiated. Though also AES-256 will be used as the encryption standard, mind that this is not the same encrytion process that we should always have in place when targeting an S3 bucket from your backups. Encryption with VBR is already done on the job level.

An attacker could thus encrypt your backup data another time - with a key he defines.

Problem is: this “customer provided key” is only processed during encryption, but not stored inside the S3 stack. Only an HMAC is stored - which is not sufficient for a decryption.

There already is a ransomware by the name of Codefinger which is leveraging that mechanism. An attacker with enough time can even circumvent immutability via object-lock by just waiting until everything in locked state is also already encrypted.

 

So, what can be taken out of that to mitigate the risks to our repos?

  • Don’t expose access keys and secrets for your buckets. Quite trivial, but very important.
  • Limit the access to the buckets used for your backups to only the public IPs of your S3 gateway servers (ACL). See example for Wasabi.
  • Restrict the usage of SSE-C. See example for AWS:
{
    "Version": "2024-10-17",
    "Id": "PutObjectPolicy",
    "Statement": [
        {
            "Sid": "RestrictSSECObjectUploads",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::my-s3-demo-bucket/*",
            "Condition": {
                "Null": {
                    "s3:x-amz-server-side-encryption-customer-algorithm": "false"
                }
            }
        }
    ]
}		

 

A feature request to Veeam would be to validate those settings or even be able to determine malicious activity inside your buckets.

For the time being: heads up!

15 comments

vAdmin
Forum|alt.badge.img+2
  • Influencer
  • 168 comments
  • January 15, 2025

Thank you ​@Michael Melter for sharing this great idea.

Does the Azure Storage Account also vulnerable with this type of attack as well?


Jean.peres.bkp
Forum|alt.badge.img+8

Thanks for sharing!


Michael Melter
Forum|alt.badge.img+12
  • Author
  • Veeam Legend
  • 144 comments
  • January 15, 2025

 

vAdmin wrote:

Thank you ​@Michael Melter for sharing this great idea.

Does the Azure Storage Account also vulnerable with this type of attack as well?

Methods seem to be there: Data encryption models in Microsoft Azure | Microsoft Learn

But I haven’t checked in detail.


Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8459 comments
  • January 15, 2025

Some great examples of securing S3. Thanks for sharing Michael. 👍


marco_s
Forum|alt.badge.img+8
  • Influencer
  • 369 comments
  • January 15, 2025

A really critical point..thanks ​@Michael Melter for sharing!


AndrePulia
Forum|alt.badge.img+6
  • Veeam Legend, Veeam Vanguard
  • 333 comments
  • January 15, 2025

@Michael Melter will the loss protection enable be an option to mitigate that? what do you think? 


Michael Melter
Forum|alt.badge.img+12
  • Author
  • Veeam Legend
  • 144 comments
  • January 17, 2025
AndrePulia wrote:

@Michael Melter will the loss protection enable be an option to mitigate that? what do you think? 

I touched that in the article. For the most recent data: yes. But once someone gains this access, he could just wait until the object-lock times out and the encryption is complete for all the data. By these methods you can instruct the S3 stack to encrypt data by itself (and throw away the key afterwards)…

I’ll do some more investigation on our implementations and will follow up if I gain more insight.


AndrePulia
Forum|alt.badge.img+6
  • Veeam Legend, Veeam Vanguard
  • 333 comments
  • January 17, 2025
Michael Melter wrote:
AndrePulia wrote:

@Michael Melter will the loss protection enable be an option to mitigate that? what do you think? 

I touched that in the article. For the most recent data: yes. But once someone gains this access, he could just wait until the object-lock times out and the encryption is complete for all the data. By these methods you can instruct the S3 stack to encrypt data by itself (and throw away the key afterwards)…

I’ll do some more investigation on our implementations and will follow up if I gain more insight.

Thank you MIchael.


MarcoLuvisi
Forum|alt.badge.img+5
  • Influencer
  • 265 comments
  • January 17, 2025

Very interesting post ​@Michael Melter thanks !


Michael Melter
Forum|alt.badge.img+12
  • Author
  • Veeam Legend
  • 144 comments
  • January 24, 2025

Got some additional information from Wasabi: they currently do not support the "s3:x-amz-server-side-encryption-customer-algorithm" condition. So the workaround I mentioned for AWS is not applicable here at the moment. However, they are working on it. I'll keep you posted. 

For now: Make sure you use ACLs to restrict your buckets to the public IPs that your backups are targeting. This will reduce your attack surface by several orders of magnitude...


AndrePulia
Forum|alt.badge.img+6
  • Veeam Legend, Veeam Vanguard
  • 333 comments
  • January 24, 2025
Michael Melter wrote:

Got some additional information from Wasabi: they currently do not support the "s3:x-amz-server-side-encryption-customer-algorithm" condition. So the workaround I mentioned for AWS is not applicable here at the moment. However, they are working on it. I'll keep you posted. 

For now: Make sure you use ACLs to restrict your buckets to the public IPs that your backups are targeting. This will reduce your attack surface by several orders of magnitude...

Thank you Michael


dloseke
Forum|alt.badge.img+8
  • Veeam Vanguard
  • 1447 comments
  • February 3, 2025

This is an interesting post and an angle of attach I hadn’t considered.  Thanks for posting!


k00laidIT
Forum|alt.badge.img+8
  • Veeam Vanguard
  • 73 comments
  • February 5, 2025

Hi ​@Michael Melter, great post! A couple things I’d like to add here. Potentially the best way to handle this for larger S3 customers is to start with reviewing AWS’ post on mitigating these types of attacks (https://aws.amazon.com/blogs/security/preventing-unintended-encryption-of-amazon-s3-objects/), focusing on leveraging Amazon Organizations and the use of Resource Control Policies (RCPs) to block use of SSE-C encryption unless your org uses it.

The other thing I would like to call out would be for Veeam to look to implemented the newer, more preferred use of temporary credentials for integrating VBR/VB365/etc. with Amazon IAM rather than the now very legacy key pair method. This is a major step forward in terms of security and should be considered.


Michael Melter
Forum|alt.badge.img+12
  • Author
  • Veeam Legend
  • 144 comments
  • February 5, 2025
k00laidIT wrote:

Hi ​@Michael Melter, great post! A couple things I’d like to add here. Potentially the best way to handle this for larger S3 customers is to start with reviewing AWS’ post on mitigating these types of attacks (https://aws.amazon.com/blogs/security/preventing-unintended-encryption-of-amazon-s3-objects/), focusing on leveraging Amazon Organizations and the use of Resource Control Policies (RCPs) to block use of SSE-C encryption unless your org uses it.

The other thing I would like to call out would be for Veeam to look to implemented the newer, more preferred use of temporary credentials for integrating VBR/VB365/etc. with Amazon IAM rather than the now very legacy key pair method. This is a major step forward in terms of security and should be considered.

Hey Jim.

Same mitigation was mentioned in the post. Thing is, Wasabi doesn’t support this mitigation method right now. They well support the SSE-C encryption though… ;)

I don’t think VBR uses SSE-C. They encrypt client side already on the GW (AES-256), don’t they? So, disabling it shouldn’t be a problem. We use IP ACLs to restrict buckets to outgoing IPs on customers side. But Veeam should maybe look into the attack and enable VBR to maybe detect this kind of activity in buckets.


k00laidIT
Forum|alt.badge.img+8
  • Veeam Vanguard
  • 73 comments
  • February 5, 2025

Yeah Veeam is not aware of the Server Side Encryption at all, they do their own thing, but what they do use is the long term key pairs as opposed to time constrained credentials. This type of attack, like Codefinger, as you know is first using exposed keys to access the bucket, then rewrite each object with a customer provided encryption key. Of course simply using object-lock/immutability also well protects against this as well, especially with Veeam which is smart enough to check versioning of an object and simply roll back to the last version it knows it wrote to.


Comment