Veeam Oxford Style Debate - Episode 1

There is always something to debate, Geoff! ;) 

I am challenging ​@GerhardGibbs all they way from the southern side of the globe.
Fighting out of Sunny (getting quite cold) South Africa.
 

I am challenging ​@GerhardGibbs all they way from the southern side of the globe.
Fighting out of Sunny (getting quite cold) South Africa.
 

 

OMG! That looks perfect ​@GerhardGibbs !😁

​@Jonty I don’t know if I would step into the ring with glasses on 🤣. I would not so much be worried about my face (modeling career never really took off 😓 ) but more the prices of glasses, $350 Canadian dollars and I sat on a pair once too so 😄

Its not that bad. In the ring the glasses just become contact lenses🤣 at that price I would have to be blind. Luckily my eye sight was free (For now). It my hairline I am going to have to start paying for. Maybe I need to reach out to the Veeam Community in Turkey.

Hi all,

First of all, ​@Madi.Cristil , thank you and congratulations on launching this new type of community challenge.

I really like the concept. It encourages us to explore different perspectives, challenge our assumptions, and discover fresh ideas. In the end, this helps all of us become better professionals and enables us to deliver more secure and reliable solutions to our customers.

One of my core objectives is to provide high-quality solutions based on best practices ensuring that customer data remains as protected as possible.

Although I was not directly challenged, ​@Iams3le  mentioned one of my previous posts—thank you for that, Christian 😊.

As a strong advocate of the 3-2-1-1-0 rule and more recently the 3-2-1-2-0 rule (From Backup to Cyber Resilience | Veeam Community Resource Hub), I’d like to share my perspective.

If the question is:

“Are immutable backups the only effective defense against ransomware?”

My answer is: Absolutely not.

If the question is:

“Should immutable backups be part of your ransomware defense strategy?”

Then my answer is: Absolutely yes.

However, immutability alone is not sufficient.

Why Immutable Backups Are Essential—but Not Enough

Even with a well-designed 3-2-1-2-0 strategy, there is no guarantee of complete protection in every scenario.

The 3-2-1-2-0 rule is an improvement over 3-2-1-1-0 because it removes single points of failure. But does it provide absolute protection? No. It significantly improves resilience, but no single measure is foolproof.

Immutability means that backups cannot be modified or deleted for a defined retention period. This is a powerful safeguard, but it is still implemented through software—and software can contain bugs, vulnerabilities or configuration errors.

Can we trust any software-based protection 100%? Probably not.

What About Tape?

Tape often gets criticized because of its operational drawbacks, but it still has a very important role.

If tapes are ejected daily and stored securely offsite, they become truly air-gapped. In many cases, that makes them even more resilient than software-based immutability because they are physically unreachable by attackers.

Does that make tape the perfect solution? No.

But in the right environment, it can be an excellent option.

True Immutability

Hardened repositories provide a strong level of protection, but they are not invulnerable.

If an attacker gains access to management interfaces such as iDRAC, iLO, or IPMI, they may be able to destroy the repository at the hardware level.

This is why I consider solutions such as ObjectFirst to be particularly compelling. Their implementation of the Eight Eyes Principle provides a much stronger form of immutability by requiring multiple authorized parties to gain root-level access.

But even then, immutable storage is still only one part of the broader defense strategy.

Security Is About Layers

Backups are your last line of defense, not your first.

A complete ransomware protection strategy should also include:

• Firewalls
• Antivirus and endpoint detection
• Network segmentation
• Multi-factor authentication
• Least privilege access
• Continuous monitoring

Implementing immutable backups without these proactive security measures is like installing a state-of-the-art alarm system while leaving your front door unlocked.

The Insurance Analogy

Cybersecurity and data protection should be viewed much like insurance.

The more risks you want to mitigate—ransomware, insider threats, hardware failure, natural disasters—the more layers of protection you need. As your resilience increases, so does the complexity, management, knowledge and cost.

The key is finding the right balance for each customer’s requirements, risk tolerance and budget.

That is why I always listen carefully to what customers value and what concerns them. My goal is not to push a single solution but to explain the strengths and trade-offs of each approach.

My conclusion

Immutable backups are a critical component of any modern ransomware defense strategy.

But they are neither the only defense nor a sufficient defense on their own.

A robust strategy combines immutable and/or air-gapped backups with strong preventive security controls and operational best practices.

In that spirit, I would like to challenge ​@Michael Melter 

Thanks for bringing me into the game, ​@Nico Losschaert. 😁

Great concept, ​@Madi.Cristil. We will though need something more controversial next time I guess… 😎

I’m of course also team “AGAINST”. Despite immutability being one of the key aspects of ransomware resilience, it can never be the only one.

The colleagues already brought all the arguments that I share, which I will not repeat. Just a single one: If you don’t realize being “pwned” already, you’re backups will be out of the immutability window once you need them.

Also - as a big fan of every modification of cheese - would I want to bring up the so called Swiss-Cheese-Model.

If you want to prevent something bad from happening - no matter if in aeronautics, healthcare or IT - you have to accept that every layer of security has holes in it. By stacking several layers the probability rises (but will never be 100%), that you will hopefully never have a path through only holes left.

Maybe immutability is a really thick slice of good Emmental cheese in here - but should not be the only one.

I’d be interested what ​@MatzeB has to say to this? Wouldn’t a good and immutable NetApp snapshot be all you need for final salvation? 😉

Thanks for bringing me into the game, ​@Nico Losschaert. 😁

Great concept, ​@Madi.Cristil. We will though need something more controversial next time I guess… 😎

I’m of course also team “AGAINST”. Despite immutability being one of the key aspects of ransomware resilience, it can never be the only one.

The colleagues already brought all the arguments that I share, which I will not repeat. Just a single one: If you don’t realize being “pwned” already, you’re backups will be out of the immutability window once you need them.

Also - as a big fan of every modification of cheese - would I want to bring up the so called Swiss-Cheese-Model.

If you want to prevent something bad from happening - no matter if in aeronautics, healthcare or IT - you have to accept that every layer of security has holes in it. By stacking several layers the probability rises (but will never be 100%), that you will hopefully never have a path through only holes left.

Maybe immutability is a really thick slice of good Emmental cheese in here - but should not be the only one.

I’d be interested what ​@MatzeB has to say to this? Wouldn’t a good and immutable NetApp snapshot be all you need for final salvation? 😉

Thank you ​@Michael Melter ! I know, this is just first episode ;) ​@DChiavari promised to help with some good challenging topics , so stay tuned ;) 

AGAINST

As I've written on several occasions on my blog, immutability alone is not a silver bullet. This is an important part of a “Radical Resilience” strategy, but it is wrong to see it as the only protection. It only gives people a false sense of security. If we accept that immutability is just a passive vault, the next step is engineering active resilience, which plays out like this in production:

1. It doesn't stop double extortion

Modern ransomware doesn't just encrypt data; it also steals it. If you have a backup protected by S3 Object Lock or on a hardened XFS repository, your data will be saved. However, this does not stop attackers from threatening to publish sensitive documents. The regulatory (GDPR) and reputational damage happens anyway. (I dive deeper into this here: Backup security: Why go with on-premises object storage?)

• Check & Mitigation: There is no software out there that can restore data after it's been illegally taken. We need to stop this from happening. Veeam with the Inline detection detectimalware during backup and monitoring suspicious activity in the file system. With EDR/XDR tools and strict microsegmentation can stop unauthorised processes and prevent the spread of attacks before data is fully stolen.

2. The sleeper ransomware 

Attackers often stay hidden on the network for months. If you make immutable backups of an environment that's already been compromised, you'll end up with a backup you can't use. If you try to restore it, the attack will come back to life. As I highlighted when analyzing the advantages and disadvantages of immutability from an audit perspective, backups must evolve from a simple "lifeline" to an active ecosystem capable of understanding data anomalies.

• Check & Mitigation: You can't restore a VM just because the backup file is immutable. Mitigation means having the right tools to clean things up, like Veeam Secure Restore, which mounts the backup and runs an active antivirus scan before connecting the VM to the network. It's also really important to use YARA rule scanning. When the incident response team identifies the malware strain, they need to inject the YARA signature into Veeam to analyse immutable points and find the last truly clean state. Using the Veeam Incident API to let the EDR system report compromised restore points directly is a real game changer.

3. The trade-off between security, costs, and RTO

One of the biggest challenges in this field is getting customers to understand that "immutable" isn't free. The cost per terabyte needs to be kept in line with what people expect. For example if attackers destroy the virtualisation infrastructure and Active Directory because there's a lack of network segmentation, the RTO will still be really low thanks to the immutability. You cannot restore petabytes of data to thin air. (I addressed the complex choices of this balancing act here: When Every Terabyte Counts: How I Chose Between XFS/ReFS and S3 Object Lock for Backups)

• Check & Mitigation: Immutability ensures the availability of restore points for recovery, but automated tools are needed. Veeam Recovery Orchestrator (VRO) replaces manual disaster recovery by automating runbooks and starting virtual machines in the correct dependency order. Additionally, a clean room is required to isolate recovered workloads for forensic analysis.

Immutability is really important, but it isn't the only thing that needs to be thought about. It needs to be used with active defences, recovery orchestration, segmentation, and proactive threat hunting.

​@PeteSteven what do you think? 

FOR! Great arguments so far. Let me borrow some comments from ​@Nico Losschaert! By the way, I learnt a lot form your addition too and I hope to see you engage more often. 

> This is a powerful safeguard, but it is still implemented through software—and software can contain bugs, vulnerabilities or configuration errors.

I really do not rate Linux based immutability due to numerous issues such as the reliance on NTP to determine when the retention period expires and attackers can sometimes try to hack the network clock to trick the system into thinking in the future.

Before the adoption of immutable storage architectures (e.g., hardened Linux repositories and object storage with WORM/Object Lock), ransomware attacks frequently succeeded despite layered security controls, leading to a sharp increase in ransom payments.

This was largely because attackers could compromise or delete backup data, eliminating recovery options. However, with the introduction of immutability, organizations gained the ability to maintain tamper‑proof backups that cannot be altered, encrypted, or deleted, even by attackers with elevated privileges. As a result, the attacker’s leverage is significantly reduced, leading to a measurable decline in ransom payments and increased ability to recover without paying.

> Hardened repositories provide a strong level of protection, but they are not invulnerable. If an attacker gains access to management interfaces such as iDRAC, iLO, or IPMI, they may be able to destroy the repository at the hardware level. This is why I consider solutions such as ObjectFirst to be particularly compelling

I love the fact Oject First was acknowledged here. When it comes to stopping ransomware from destroying your ability to restore, immutability is the only mechanism that works. I remember we had this discussion on discord and was not properly dealth with. With OOTBI, the above concerns are not present. If the backup can't be deleted, modified, or encrypted even by a compromised domain admin account, then the attacker loses their ultimate leverage. Immutability isn't a replacement for a firewall or MFA as we mentioned yesterday; it is the final safety net just like ​@Geoff Burke mentioned yesterday. It is a life insurance!

We have spoken extensively about different security safeguards, and Veeam Software should not be left out of this discussion. For example, Veeam incorporates capabilities such as inline malware detection and entropy analysis to identify suspicious data patterns before or during the backup process. These mechanisms can help detect potentially compromised or “poisoned” backups by flagging abnormal encryption-like behavior commonly associated with ransomware activity.

I would consider this another important safeguard within a broader cyber resilience strategy. While immutability protects backup integrity after data has been written, proactive detection capabilities help organizations identify threats earlier in the backup lifecycle, further strengthening ransomware recovery readiness.

> Tape often gets criticized because of its operational drawbacks, but it still has a very important role. If tapes are ejected daily and stored securely offsite, they become truly air-gapped. In many cases, that makes them even more resilient than software-based immutability because they are physically unreachable by attackers. Does that make tape the perfect solution? No.

Tape-based air-gapped backups still play an important role in cyber resilience, particularly because physically disconnected media cannot be directly reached over the network once properly stored offline. However, air-gap protection relies heavily on operational processes like you mentioned, including correct tape rotation, secure transportation, storage handling, and disciplined recovery procedures. Human error, delayed rotations, misplaced media, and operational gaps can all reduce the effectiveness of an air-gapped strategy. Also, a physical air-gap introduces massive, business-killing friction when you actually need to recover!

Immutability addresses the problem differently. Instead of relying on physical disconnection, immutable storage enforces protection directly at the storage layer by preventing backup data from being modified or deleted during the retention period, even by privileged administrators. This protection remains continuously active while the backup system stays online.

The key distinction is that air-gap primarily provides isolation, while immutability provides deterministic enforcement of backup integrity. In modern ransomware attacks where backup repositories are deliberately targeted, immutability ensures that attackers cannot tamper with protected recovery points even after gaining administrative access to the environment.

Tape air-gap remains valuable. But immutability delivers faster operational recovery, and more consistent enforcement in day-to-day ransomware defense strategies. Therefore, I would say, Immutable, is the single most important ransomware defence.

Immutable Storage Evolution

To understand why immutability is the ultimate game-changer, we have to look at the history of ransomware. Before the introduction of dedicated immutable storage such as the Linux Hardened Repositories and Object First appliances. There was a sharp rise in devastating ransomware payouts despite organizations having multi-layered perimeter controls in place.

Why? Because attackers realized that network security could eventually be breached via credential theft or unpatched vulnerabilities. Once inside, their primary objective wasn't to encrypt production right away; it was to find and completely destroy the backups first. According to Veeam’s research, threat actors target backup repositories in 89% of attacks, and historically, they successfully compromised them in the vast majority of cases. When traditional backups were deleted or encrypted, organizations were forced into a corner and had no choice but to pay the ransom. This is what the debate is all about from my point of view!

However, since the widespread adoption of immutability, we have seen a significant, documentable drop in overall ransom payments. Industry data shows that a record 64% of organizations now flatly refuse to pay ransom demands

The reason for this shift is a direct change in leverage. Immutability takes away the attacker’s 'kill switch.' When an organization can confidently restore from an un-erasable, un-modifiable backup appliance like Object First, the hacker loses their leverage. Threat actors can no longer force a payout by destroying the last line of recovery. This historical shift proves that while perimeter security tries to keep the bad guys out, immutability is what actually breaks the ransomware business model by ensuring you can always walk away without paying."

Some quotes and references below:
1: "Ransomware is changing how organizations approach backup and data protection. As backup infrastructure faces more threats, IT teams recognize that immutability is key to protecting data": https://objectfirst.com/newsroom/press-releases/esg-research-finds-immutable-backup-storage-following-zero-trust-as-the-best-defense-against-ransomware/
2: https://objectfirst.com/newsroom/press-releases/object-first-research-93-of-it-professionals-say-immutable-storage-is-essential-to-protect-against-ransomware-attacks-on-backup-data/
3: https://app.stationx.net/articles/ransomware-statistics
4: "Organizations are resisting ransomware demands due to the lack of conviction that they’ll get their data back—hackers often fail to release data even after payment. To counter this, many have strengthened their incident response strategies, finding vendors that provide immutable backups to ensure data protection and recovery without paying a ransom.": https://objectfirst.com/blog/summary-of-the-veeam-ransomware-trends-report-2025/
5: https://www.acronis.com/en/tru/posts/immutable-backups-the-critical-gap-between-backup-success-and-real-recovery-readiness/
6: "... organizations are investing in tested, immutable backup systems that allow them to restore operations without rebuilding from scratch": https://cnicsolutions.com/statistics/ransomware/ransomware-recovery-statistics-2026/
 

I am challenging, Edwin ​@Viperia to this!

FOR!

> As I've written on several occasions on my blog, immutability alone is not a silver bullet. This is an important part of a “Radical Resilience” strategy, but it is wrong to see it as the only protection. 

​@Andanet, as we have highlighted, immutability alone is not a silver bullet. It is a critical component of a broader resilience strategy, but it should not be viewed as the sole layer of protection. However, within the context of ransomware recovery, immutable storage remains one of the most effective controls for ensuring backup integrity. It uniquely guarantees that recovery data cannot be altered or deleted during the retention period, even in the event of administrative compromise. This is why immutability is best understood not as a replacement for other controls, but as a foundational recovery assurance mechanism within a defense-in-depth model. Security controls such as monitoring, MFA, segmentation, and malware detection reduce risk exposure, but immutability ensures that recovery remains possible when those controls fail. In that sense, defense-in-depth is not a reason to downplay immutability, but the reason immutability is essential.

> Modern ransomware doesn't just encrypt data; it also steals it. If you have a backup protected by S3 Object Lock or on a hardened XFS repository, your data will be saved. However, this does not stop attackers from threatening to publish sensitive documents.

Modern ransomware has indeed evolved beyond encryption-only attacks, with many threat actors now exfiltrating data and using it for extortion through leak threats. However, this does not diminish the role of immutable backups; it clarifies their scope.

Immutable storage mechanisms such as S3 Object Lock and hardened repositories primarily protect backup integrity and ensure recoverability by preventing deletion or tampering of recovery points. They are designed to guarantee operational restoration capability, even under full administrative compromise.

Data exfiltration is a separate attack vector that targets confidentiality rather than availability. This requires complementary controls such as data loss prevention (DLP), encryption at rest and in transit, access governance, and monitoring/insider threat detection.

Therefore, the presence of exfiltration-based ransomware does not invalidate immutability; it reinforces the need for a layered security model where immutability ensures recovery, while other controls address confidentiality and prevention.

FOR! To properly evaluate ransomware resilience controls, it is important to separate security mechanisms by their primary function such as prevention, detection, containment, and recovery assurance. The following matrix outlines common safeguards and highlights their specific strengths and limitations. While most controls reduce the likelihood or impact of an attack, they do not all provide the same guarantee of data recoverability. Immutable storage stands out because it enforces backup integrity at the data layer, ensuring recovery points cannot be altered or deleted during the retention period, even under compromise.

On Linux XFS, immutability is an OS control. If root on the repo gets compromised, that boundary is compromised too, a process with cap_linux_immutable can clear the -i flag. Dropping cap_linux_immutable is absolutely a good hardening step, but it only protects the process tree where it is applied, not every possible privileged path on the box. Basics still matter: keep the Linux repo patched from trusted repositories, limit access and isolate it. Did someone say Ootbi? 🙂 This is another reason I think it is worth a look. We all love v13, but it is still pretty fresh, and there are a lot of roll-your-own Linux repos out there, including some that started life from older ISO builds and may not be as patched or hardened as people think. 👉👈