• EN

Historical archive of deleted VM's.

Scott
Userlevel 7
Badge +8

I was wondering if anyone else does this.

When we delete a VM, I often have someone come ask me where it was 6 months to a year later and sometimes there is something unnoticed that is broken. 

Due to the amount of storage we use here (PB’s) I tend to not want to keep all these old VM’s powered off, or even on the Veeam SANs as they take up a ton of space for something that most likely isn’t needed.

 

To resolve this I created a media pool on both tape libraries that has a retention of 2 years. When I power the VM off for the final time I do a Veeam zip, then add it to a file to tape job at both sites.

In the file to tape job I just grab the VBK from the repo and then can remove the Veeam ZIP when it’s done.  Even though it’s not 3-2-1 technically the VM’s are requested to be deleted anyways. If I lost them it wouldn’t be critical. Having them on tape is cheap, and people are very happy when they can get their data.

 

 

 

 

 

 

30 comments

Scott
Userlevel 7
Badge +8

 

 

 

Hannes is not taking us seriously in the forum! 😀

HAHA

 

Quite the opposite actually. I have had amazing help from those guys. 

JMeixner
Userlevel 7
Badge +17

I do a similar approach.

Several colleagues asked for a deleted VM some weeks after the deletion. So. I do a VeeamZIP of them and keep it for 3 months normally. I don’t copy them to tape, because we don’t have that much deleted VMs, I keep them on a separate disk storage. You can set an automatic deletion date on the VeeamZIPs, then you don’t have to do housekeeping for them.

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
Chris.Childerhose
Userlevel 7
Badge +20

VeeamZIP is your friend for sure.  We do this with every cleanup of any VMs is VeeamZIP them first and store them off on to different storage.  Keeps us sane so we can do that restore if needed. 😋

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
MicoolPaul
Userlevel 7
Badge +20

When I was last internal IT I didn’t have tape infrastructure (rotated USB’s for airgap). As a result I just did a final VeeamZIP of the powered off VM and commited that to a separate folder on a SOBR extent. Once it’s not ‘required’ anymore and the thresholds for retention were met, it was just having that copy for ‘peace of mind’ in the event that exactly what you said would happen. Nice idea doing it to tape, alternatively now people could also throw it into some archive tier cloud storage (about to be a lot easier with v12 as well!) and just leave it there for a while.

 

I tended to do this because the business’ retention period was quite short (months not years), so it would age out of backups quite quickly.

 

Good discussion point 👍

Michael Paul - #VeeamLegend | #VeeamVanguard | Veeam Certified Architect | VMware vExpert | https://micoolpaul.com | Twitter: @MicoolPaul | Mastodon: @micoolpaul@masto.nu | Bluesky: @micoolpaul.bsky.social
Iams3le
Userlevel 7
Badge +9

Great point @Scott! You are doing it correctly with Veeam ZIP. Here is a fantastic guide from @Nico Losschaert on this topic. There are other ways to achieve this without Veeam as well. 

[Microsoft MVP | Veeam Legend 5* | VMware vExpert 4* | Cisco Champion 3* | Object First Ace]
Nico Losschaert
Userlevel 7
Badge +11

Great point @Scott! You are doing it correctly with Veeam ZIP. Here is a fantastic guide from @Nico Losschaert on this topic. There are other ways to achieve this without Veeam as well. 

Thx @Iams3le 😍

#Veeam Legend 5* | VMCA 2* | VMCE 3* | VMTSP+ 2* | VMSP 2* | DCIE 2* | Loves Best Practices | Twitter : @NLosschaert
Iams3le
Userlevel 7
Badge +9

Great point @Scott! You are doing it correctly with Veeam ZIP. Here is a fantastic guide from @Nico Losschaert on this topic. There are other ways to achieve this without Veeam as well. 

Thx @Iams3le 😍

You are welcome😊

[Microsoft MVP | Veeam Legend 5* | VMware vExpert 4* | Cisco Champion 3* | Object First Ace]
regnor
Userlevel 7
Badge +14

Storing such VMs/archives on a tape sounds like a perfect solution. As an alterantive with v12 and it’s direct to object storage capability, I would create a one-time job and upload such VMs to an object storage of choice; most likely Wasabi.

Max M. | Systems Engineer @Veeam | VMCE/VMCA | former Legend & Vanguard
H
Userlevel 5
Badge +2

Hannes is not taking us seriously in the forum!  :-)

 

hmm, why do you think so? The first request can make sense, but it adds complexity with hard to estimate benefit and currently only one customer asking for it. So let’s see how many others ask for features like that

the second thing is a different topic / technology and it’s confusing for every reader to have two topics / discussions mixed in one thread :-)

Scott
Userlevel 7
Badge +8

Great point @Scott! You are doing it correctly with Veeam ZIP. Here is a fantastic guide from @Nico Losschaert on this topic. There are other ways to achieve this without Veeam as well. 

Very similar. I use the EXPORT too 

marco_s
Userlevel 7
Badge +7

The best could be a feature mixing “remove deleted data after” and SOBR: if we use capacity/archive tier, Veeam can let use to choose if we would automatically offload a deleted item e retain until the specified days set in the maintenance tab..what do you think?!

Marco Sorrentino - Italy | System Engineer | VMCE & Veeam Lover
Chris.Childerhose
Userlevel 7
Badge +20

@marco_s - that is a good suggestion and would recommend adding it to the Veeam Forums - https://forums.veeam.com/

Also, Tape is another great solution for this as well as the direct to object.  To me the direct to OBS is better since tapes would need to be tested more frequently to ensure it does not get corrupted.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
Scott
Userlevel 7
Badge +8

I have already requested both the VM backup, and the ability to do a similar thing for files in the forums a while back, but more requests can actually help make things happen.

 

A big one for us is our retention. We have to keep 100’s of TB’s of data for 60-100 years.    I’d love to “archive” files to 2 locations, (tape and Wasabi) then delete it off of the production file share. 

 

Obviously Veeam don’t want to be responsible for the deletion so if that is manual, or a script on our end I’d understand that. but Having some sort of file tree of my “archived” files under another menu would be so good. 

 

This is something we would invest in VUL licenses for even. I don’t need full file shares backed up with NAS backup as we can’t afford that vs regular Veeam backups, but this non prod stuff is costing a ton in storage for data that is not used often.

 

I checked on a few of our 30TB servers and the change rate is a few gig a day lol. another one was in the MB range.

 

 

marco_s
Userlevel 7
Badge +7

I was joking Hannes! :)

In any case, the request is not for me or for a specific person, but it is to improve the product, so it is for everyone..

We are all here because we love veeam, the goal is common!

Marco Sorrentino - Italy | System Engineer | VMCE & Veeam Lover
Scott
Userlevel 7
Badge +8

I’m in testing for Wasabi and that is the plan to 3-2-1 for archival data we have to keep for legal reasons, yet don’t require. With our 50-100 year retention policies for some stuff it gets pretty insane to keep this stuff on our SANS.

 

The Wasabi way is quicker to restore as well. AND, if you only need 1 file from the VM, you don’t have to restore the whole thing to the repository, which you do on tape. 

 

I’ve used the VeeamZIP auto removal, but our Veeam storage is always on the verge of getting tight so having a ton of VeeamZIP’s in there of not required machines is less important that PROD backups. 

dloseke
Userlevel 7
Badge +6

With our 50-100 year retention policies for some stuff it gets pretty insane to keep this stuff on our SANS.

 

Catching up on some old posts/tabs I forgot about, but I have to ask...what kind of data requires a 50-100 year retention policy?  The only thing I can think of is government records, like property records and such….

Derek M. Loseke, Senior Systems Engineer | Veeam Legend 2022-2023 | VMSP/VMTSP | VCP6-DCV | VSP/VTSP | CCNA | https://technotesanddadjokes.com | @dloseke
Scott
Userlevel 7
Badge +8

Well, it depends of the need if necessary i will quickly discuss about it with the requestor. My BaaS has 6 months retention on tape for Production so normally that’s enough. If duly justified we can make an exception for longer retention with a Veeam Zip like suggested before :)

Lucky. the 60 year and 100 year legal requirements on some of the large stuff I have is crazy. And because of that everyone figures they have to save EVERYTHING. 

BertrandFR
Userlevel 7
Badge +8

It was just my POV, it depends on many factors (opinion, internal organization, use cases). I'm not a huge fan of using backup to archive when depending on the use case it can be managed in a database.

I didn't say that it was your case but I have often come across these cases. I did not want to create any misunderstanding :)

marco_s
Userlevel 7
Badge +7

I have already requested both the VM backup, and the ability to do a similar thing for files in the forums a while back, but more requests can actually help make things happen.

 

A big one for us is our retention. We have to keep 100’s of TB’s of data for 60-100 years.    I’d love to “archive” files to 2 locations, (tape and Wasabi) then delete it off of the production file share. 

 

Obviously Veeam don’t want to be responsible for the deletion so if that is manual, or a script on our end I’d understand that. but Having some sort of file tree of my “archived” files under another menu would be so good. 

 

This is something we would invest in VUL licenses for even. I don’t need full file shares backed up with NAS backup as we can’t afford that vs regular Veeam backups, but this non prod stuff is costing a ton in storage for data that is not used often.

 

I checked on a few of our 30TB servers and the change rate is a few gig a day lol. another one was in the MB range.

 

 

Hannes is not taking us seriously in the forum! 😀

Marco Sorrentino - Italy | System Engineer | VMCE & Veeam Lover
Scott
Userlevel 7
Badge +8

With our 50-100 year retention policies for some stuff it gets pretty insane to keep this stuff on our SANS.

 

Catching up on some old posts/tabs I forgot about, but I have to ask...what kind of data requires a 50-100 year retention policy?  The only thing I can think of is government records, like property records and such….

 

I work for government, and lots to do with things such as policing, homicides etc. 

Scott
Userlevel 7
Badge +8

If you charge for the service, why not good for you, but it is abnormal if the business application uses the backup as an archive. There is a small problem of application architecture.

So the IT department and team that does the backups, file servers, storage, servers shouldn’t be in charge of the data archives and retention? Even though we use the backup software of Veeam, TSM and cloud storage, and no one else even has credentials to do so?

We are not an MSP, this is our data. The law is that we have to keep it. we are paying for this, not charging. 

 

It’s quite normal in our circumstances to have crazy retention. 

 

I just want it off prod as easy as possible and into Wasabi and tape as the chances of needed to restore it get smaller every year. It still doesn’t change it’s mandated by law. 

 

 

 

OK, my experience is, if you charge internally for the storage consumption, then are suddenly many hard requirement not so hard anymore… 😎

And yes, I agree, some legal requirements make no really sense. perhaps you have the date after all these years, but do you have machines and software to read them and work with them?

Funny you asked this. I said the same thing today. In 40 years will Veeam version 30 be able to read Windows server 2016? who knows. Will that file type of that video still work? 

 

Heck it’s not been too long since VHS and it’s hard to find a VHS player these days, or a Betamax. 

 

However, Those are things I ask to prove a point, but at the end of the day I am told to keep and archive the data for legal reasons. It doesn’t state I have to be able to access the files. Most likely the IT department will be tasked with that too, but in reality, I'll be retired in 20 years and this data will be stale. there is a large chance most of it is garbage, but not my place to say. 

 

No one said all laws made sense, or were even productive.  I have been pushing for retention policies though and getting a BIT of traction. The price tags on PB’s of disk, cloud, archive, backup, etc have helped.

 

 

JMeixner
Userlevel 7
Badge +17

Unfortunately you are right that not all regulations make sense.

And even more unfortunately it is expected in Germany that these data is readable in 30 years. I am very sceptical about this….

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
BertrandFR
Userlevel 7
Badge +8

It is an interesting debate that we have, we can see differents opinions/use case and regulations of each country. At the European level, retention is more regulated since the emergence of GDPR. There may be subtleties for the justice and police part for example, the majority of use case could be done in DB by business apps. I insist on it but it was a nice battle with the projects to convince them that the backup should not be in adhesion for this archiving, specially when they don’t pay for the backup service.

Example for logs collecting and sequestration, i prefer to handle this by a distinct part. Example you can push you data directly to an object storage with tiering and ensure legal hold by the bucket configuration...

If necessary it seems to me to have a wise discussion with the DPO, without forgetting that one moment it will be necessary to evoke the technical aspect and especially financial constraints.

BertrandFR
Userlevel 7
Badge +8

Well, it depends of the need if necessary i will quickly discuss about it with the requestor. My BaaS has 6 months retention on tape for Production so normally that’s enough. If duly justified we can make an exception for longer retention with a Veeam Zip like suggested before :)

BertrandFR
Userlevel 7
Badge +8

Wow, it doesn’t make sense… Sometimes reason does not prevail over intriguing customer demand, i don’t want to be in a brain of a DPO who asks for this.

Comment


Terms & Conditions