Just wanted to share a VM recovery tip with you all I recently discovered when needing to recover a VM no longer being processed by a VBR Backup Job it was in due to Deleted VM Maintenance Schedule. To assist with my recovery, you have to be utilizing Storage Integration technology with Veeam. My specific environment uses Nimble Adaptive Flash Arrays, so be sure to understand how your Storage Vendor’s snapshot technology works before assuming my “tip” will apply to your environment.
Let’s assume you’re backing up VMs with vSphere Tags and during tag maintenance you mis-apply a tag, or forget to (re) apply a tag to a VM needing to be backed up. Or maybe you use vSphere “containers” to back up your VMs (i.e. Datastores, VM Folders), and you mistakingly move a VM out of the Container and the VM is no longer getting backed up. As a good Backup Admin does, such that you are, you review your VBR jobs daily to verify they succeeded:
BOOM! Well done, Backup Admin! But, you weren’t being as thorough as you should and aren’t actively looking at the backup job’s job history details which would show you the VM you mistakingly mis-tagged or removed from vSphere Container(s) are “no longer being processed by the job and to verify if this is intentional”. Low and behold, 15 days later, this VM has an issue to where you need to restore a guest file or whole VM image. You look under the VBR Home pane > Backups section > Disks, and expand the VBR Backup Job the VM used to be in and you do not see the VM because the Maintenance tab setting for Backup jobs has a default 14 days to retain “deleted” VMs. Oh no! Now what are you going to do??
Well, your Storage Array Snapshot functionality you integrated into VBR could be what recues the day for you. Although you may have configured VBR to perform your array snapshot orchestration, if your Storage Vendor handles snapshotting functionality like Nimble, you then may have longer retention of VM backups than what a VBR Backup Job Retention Policy is configured for. Let me explain…
With Nimble Arrays, when you configure your Volumes to be used for snapshotting within Veeam, you have to place your Volumes in what’s called a Volume Collection in Nimble, which is basically just a “snapshot schedule container”. But, this Volume Collection “container” can not have a blank snapshot schedule, which you would be ok with because you’re configuring Veeam to handle snapshotting. But, Nimble doesn’t allow this. As such, Veeam recommends configuring the Volume Collection to snapshot the least amount of time as is configurable. With Nimble, this is 1 week out. You can also determine how many snapshots (and replications, if you also configure this setting to a downstream/off-site array) are kept:
Depending on the snapshot cleanup maintenance the array performs, you more than likely will have a snapshot of your Volumes for longer than the default VBR Backup Job Delete VMs schedule of 14 days, if you configured your Volume Collection to retain a minimum of 2 snapshots. YAY!
To check if you still have a backup of the VM you’re needing to restore, go to the Home pane in VBR > Backup section > Snapshots. You should see a list of Volume Snapshots in the main “working area” on the right. Expand the Volume Snapshot the VM is on and see if the VM is there. If it is, you can then rt-click it and select a recovery task to run.
Hopefully this little tip helps you in the future. It’s so beneficial to have numerous ways to be able to recover your data when needed.
Great idea; if everything else fails you'll still have the storage snapshots as a last resort.
How do you handle the ‘removed deleted items’ option? I normally leave it disabled (default) as I'm afraid of deleting restore points in an error. If it's necessary I enable it for a certain time and disable it afterwards.
Oh no....enable that bad boy. ALWAYS have a failsafe. Basically, that's what this post is with regards to the "Delete VMs After..." setting.
Great tip! Always a good idea to create as much options as possible for restoring! Furthermore with Nimble you should see no performance degradation with active snapshots.
Breaking News: Don’t call it Nimble any more, call it Alletra!
Not happy about Alletra tbh. Maybe it’s just solely a name change. But the way HPE has done everything they’ve touched in their history, I doubt it’ll be just that. It won’t be the same, sadly.