Skip to main content
  • EN

It is always interesting, when thinking of special situations, to ask: What happens, when …

Here are the answers to at least a few of these questions :grin: - for immutable backups in VBR v11:

What happens, when…

… removing immutable restore points?
Task starts and ends up with errors. No restore point is removed.

Warning when trying to remove immutable files

… hacker changes date/time on Veeam server to remove files with next backup run?
Excellent question! But it will not work! Configured period of time is checked at Linux server, not at Veeam B&R Server! So a job running one year in the future will create new files, it cannot delete old ones.

… enabling feature at existing repository, not compatible with hardened repository?
Veeam checks if there are backup jobs configured with an incompatible backup chain. If found, error is shown:

Error with not compatible backups

… changing a backup job writing to immutable repository?
When backup chain is changed to an incompatible one, error is shown:

… days for immutability is higher than the number of restore points?
Restore points are kept to the end of the period, defined for immutability. Notice that there is an information message in job log:

Info that files are still immutable

… immutability turned on on repository containing restore points?
In this case immutable flag is set on all files, according immutability period.

… immutability turned off on repository containing immutable backups?
In this case, immutable flags are kept. That makes perfect sense. Otherwise a hacker with backup administrator permissions could disable hardened repository to delete backup files. Notice: new backups are not immutable after disabling the feature.

… trying to remove restore points in GUI on files with immutable flag manually removed?
Interestingly files will not be deleted! Probably because Veeam knows, files should be protected. There will be more details on that after the release of v11.

Cannot remove files even immutable flag is manually removed

… use immutable repository as target for copy job without GFS?
Backup copy jobs requires GFS when copied to immutable repository. If no GFS is set up, error appears.

 

I wouldn't say it's the easy way, but yes this would work. While I would say 7 days is a rather short time frame for immutability, at least you would lose your current backups.

That's why you still need to secure your environment and should monitor all configuration changes. Immutability is only a part of the solution.

So the easy way to attack a inmutable copy is:

  • Take control of Veeam server.
  • change the repository config. (remove inmutability)
  • wait the inmutability time (let’s say 7 days)
  • You will be able to delete restore points from Veeam console… and final user has had no warning.

 

Am I correct?

you would still have to wait for the immutability time to expire, so if I put 160 days…. The lock is on the object store so Veeam and windows can’t change it.

So the easy way to attack a inmutable copy is:

  • Take control of Veeam server.
  • change the repository config. (remove inmutability)
  • wait the inmutability time (let’s say 7 days)
  • You will be able to delete restore points from Veeam console… and final user has had no warning.

 

Am I correct?

you would still have to wait for the immutability time to expire, so if I put 160 days…. The lock is on the object store so Veeam and windows can’t change it.

Now if you got console on the linux server then that is not good. If you are running your object storgage in containers in orchestrated environment then we add yet another level of difficult into the equation for the bad guys ;)

Also to add, if you’re using Veeam ONE (which, you really should be!), there’s built in alarms for Immutability state and Immutability change tracking. So this would drive your warnings.

So the easy way to attack a inmutable copy is:

  • Take control of Veeam server.
  • change the repository config. (remove inmutability)
  • wait the inmutability time (let’s say 7 days)
  • You will be able to delete restore points from Veeam console… and final user has had no warning.

 

Am I correct?

you would still have to wait for the immutability time to expire, so if I put 160 days…. The lock is on the object store so Veeam and windows can’t change it.

Yes, I said “wait the inmutability time”… if it’s very long you will be safer… but you will need a really big disk

Also to add, if you’re using Veeam ONE (which, you really should be!), there’s built in alarms for Immutability state and Immutability change tracking. So this would drive your warnings.

that’s a good point!! thanks!

@Zucchetti Spain, correct on waiting the immutability time, I can’t find the document but I thought I saw something about not being able to change the immutability state of a hardened repository after initial deployment, I may be thinking of Object Storage, @vNote42 / @Mildur, does this ring any bells to yourselves?

@Zucchetti Spain, correct on waiting the immutability time, I can’t find the document but I thought I saw something about not being able to change the immutability state of a hardened repository after initial deployment, I may be thinking of Object Storage, @vNote42 / @Mildur, does this ring any bells to yourselves?

I have tried It and I can remove the chek in Veeam… with no warning. now i’m waiting 7 days to pass but I’m quite sure I will be able to delete my (test) backups…

 

In my opinion Veeam should warn about “this backup has changed form inmutable to non inmutable” for at least some days...

@MicoolPaul

Are you talking about disabling Immutability? Disabling immutability on an object Storage is not possible. You get an error.

 

 

Immutability on Linux Hardened Repositories can be disabled in my testlab, but Veeam One will alarm you about it.

 

Awesome post! 

@MicoolPaul

Are you talking about disabling Immutability? Disabling immutability on an object Storage is not possible. You get an error.

 

 

Immutability on Linux Hardened Repositories can be disabled in my testlab, but Veeam One will alarm you about it.

 

yes… I mean disabling it on Linux hardened repository…  Veeam One will warn me… but in my opinion it shoud be a good idea put a warning in the result email of the copy itself

 

You can it disable on a hardened repository. But the backups which are already in the repository stay immutable. Only the new ones are not immutable.

Yes, the VeeamOne alarm is very useful. But I agree, it would be useful when Veeam would warn itself, too….

@Zucchetti Spain, correct on waiting the immutability time, I can’t find the document but I thought I saw something about not being able to change the immutability state of a hardened repository after initial deployment, I may be thinking of Object Storage, @vNote42 / @Mildur, does this ring any bells to yourselves?

I have tried It and I can remove the chek in Veeam… with no warning. now i’m waiting 7 days to pass but I’m quite sure I will be able to delete my (test) backups…

 

In my opinion Veeam should warn about “this backup has changed form inmutable to non inmutable” for at least some days...

Thanks @Mildur for confirming, couldn’t remember if it was either or both, so your comments there are appreciated!

@Zucchetti Spain, just to also say, the immutability will be 7 days from the last point in the chain by the way, Veeam increments this on the dependent points in the active chain to ensure the chain is immutable to the same time so there’s no isolated incremental backups that aren’t usable.

@Zucchetti Spain , just add my two cents:

As already said, it is easily possible to disable immutability if you get backup-admin access to VBR server. This is because immutability is just a property of the repository, saved in the VBR-database. Therefore it is essential to monitor these settings. With Veeam ONE, you have a good tool to do so:

Monitor Hardened Repository with Veeam ONE v11a

 

Maybe also important to note: GFS restore points stay immutable as long as they are kept by retention-policy.

 

 

 

i’m wondering if someone deployed an hardened repo with openscap for hardening the OS? What is your experiences about it? did you encounter any inconvenience about OS hardening enable ?

BTW, i discovered this article on Redhat Website about the deployment of an hardened repository.

https://www.redhat.com/en/blog/veeam-ransomware-protection-rhel-immutable-repository

@vNote42 not sure it’s the right topic, if it’s not tell me i will move my message to another :)

Comment


Terms & Conditions