VMware vCenter Appliance: image backup is deprecated since 7.0


Userlevel 7
Badge +13

I had the discussion two times last week, so I think it is worth sharing here.

You may have noticed, VMware deprecated image level backup of its vCenter Appliance (VCSA) with vSphere 7. What does this mean? Basically, it is still supported to do image level backup and restore with VCSA 7.0. But it will not be supported in future releases. Notice that U1 and U2 are still 7.0 versions. With 7.n, with n>0, VMware will not support image restore of VCSA any more. 

From my perspective, this step was taken because problems arise when restoring this Linux appliance. I have seen many appliances requiring a filesystem check after restore or crash. When you are lucky, everything works fine afterwards. If not, you have to file a support ticket.

To avoid this, VMware offers a new way for VCSA backup. Since 6.5 it is possible to run file-based backups and restore of VCSA. Since 6.7 this can be scheduled. With this, new protocols are supported as well: FTP, FTPS, HTTP, HTTPS, SFTP, NFS, or SMB. Please correct me, if I am wrong: with vSphere 7 (or U1?) SMB Version greater 1 is supportet since.

https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vcenter.install.doc/GUID-1C73996F-8312-4BBD-A16C-B2C8FC3C0D31.html

So if you did not set up file-base VCSA backup, do so now:

  1. Connect to your VCSA on port 5480 using a browser.
    Using root or SSO-admin account
  2. Click Backup → Configure

     

  3. Set up Backup Schedule

 

To test it, you can click Backup now and enable Use backup location and user name from backup schedule

 


26 comments

Userlevel 7
Badge +17

We have the file-based VCSA backup in place since version 6.7. Works fine.

But… once there was a VSAN problem and the VCSA was not accessible anymore.

The file-based backup saves the files inside the VCSA only, not the configuration files of the VM. So we had to setup a new VCSA and restore the file-based backup.

If we had have a snapshot backup at hand we would have to restore the VM configuration files only…

Since this time we do a snapshot backup additionally to the file-based backup….

Userlevel 7
Badge +11

@JMeixner: While it is true it does not save any files “outside the VM” such as .vmx files you don’t need those to restore the vCenter Server from the file-based backup regardless. Could an Instant VM Recovery bring the vCenter back faster? Yes, absolutely, but I guess we just need to adjust to change :grinning:

Userlevel 7
Badge +17

Yes, I agree. And my case was rather special,  I guess.

We are doing the file-based backup. It is the supported method with V7.1 and later anyway.

But it is no big problem for us to do the snapshot backup additionally to have the configuration files at hand. This issue was not a VCSA problem, but a VSAN problem.

 

Userlevel 7
Badge +11

@JMeixner: I’d love to better understand what the issue was here. Are you saying because it was VCSA on VSAN it was not possible to restore the VCSA using the file-based backup?

Userlevel 7
Badge +17

VSAN had lost references to some files - the configuration files of the VCSA were among them. The files were there but VSAN did not find them. The VMDK files of the VCSA were there but it was not startable because of the missing VMX Files and so on. Restore of these files would have been enough to get the VCSA working again. We have tried this with several other VMs.

As I said, it was a special case… The VMWare admin colleagues are in discussion with VMWare about this issue...

Userlevel 7
Badge +13

VSAN had lost references to some files - the configuration files of the VCSA were among them. The files were there but VSAN did not find them. The VMDK files of the VCSA were there but it was not startable because of the missing VMX Files and so on. Restore of these files would have been enough to get the VCSA working again. We have tried this with several other VMs.

As I said, it was a special case… The VMWare admin colleagues are in discussion with VMWare about this issue...

I am afraid I do not understand your problem completely too. Was your problem that you have lost the VMX-file of your VCSA?

Userlevel 7
Badge +3

i agree with you.

Userlevel 7
Badge +13

[update]

I forgot to mention, that it is just possible to restore these file-base restore points, when you have exactly the same vCenter Version (incl. build) as ISO file.

Therefore I recommend to download full VCSA ISO - with same version ofupdate/upgrade - afterwards.

Userlevel 7
Badge +17

Yes, VSAN has lost reference to a whole bunch of files. Among others were the VMX and VMXF files of the VCSA.

 

Userlevel 7
Badge +3

[update]

I forgot to mention, that it is just possible to restore these file-base restore points, when you have exactly the same vCenter Version (incl. build) as ISO file.

Therefore I recommend to download full VCSA ISO - with same version ofupdate/upgrade - afterwards.

thanks for your update 

Userlevel 7
Badge +12

[update]

I forgot to mention, that it is just possible to restore these file-base restore points, when you have exactly the same vCenter Version (incl. build) as ISO file.

Therefore I recommend to download full VCSA ISO - with same version ofupdate/upgrade - afterwards.


And archive those ISOs on a safe place so that you can easily find them if needed. I've never had the problem with VMware but I often had to search for older Microsoft Exchange CUs and Microsoft doesn't offer them for download anymore...😅

Userlevel 7
Badge +11

Good to know! However I think is still easier do VCSA backup with Veeam B&R.

Userlevel 7
Badge +13

Good to know! However I think is still easier do VCSA backup with Veeam B&R.

Yes, backup is easier! But keep in restore in mind too!

Userlevel 3
Badge

Interesting topic. We are facing this issue as well at one of our clients. I would like to have good arguments to make some changes in the backup configurations. 

The question is not IF we need a backup but how to do it as efficient as possible. keep in mind that the backup is worthless if we can not restore. 😉

Configuration atm is as follows:
4 default vm backups based on vmware tags and with storage integration. Backup jobs are running every evening between 10 PM and 4 AM.

For some reason one backup job fails around 12 Am and I think that at that particular moment vcenter vm is creating a snapshot and the rest of the vm’s are failing within this job. Because of this reason we would like to create a new job with vcenter VM only.

is this the way to go or would wou guys suggest to do it differently?

Userlevel 7
Badge +7

i backup the VM VCSA 7 with veeam job, in combo with native VCSA backup, it is convenient because it supports several protocols. Uses nfs which can also be exposed on Windows

Userlevel 7
Badge +7

@JMeixner: While it is true it does not save any files “outside the VM” such as .vmx files you don’t need those to restore the vCenter Server from the file-based backup regardless. Could an Instant VM Recovery bring the vCenter back faster? Yes, absolutely, but I guess we just need to adjust to change :grinning:

During a disaster it happened that the VCSA restore was successfully completed, but the VCSA was misaligned because the old restore was used. We had to extract the ESxi from the cluster and re-insert it to synchronise everything.

Userlevel 3
Badge

@Link State Thank you for your graphical explenation 😃

Can you explain to me why you are backing up also by Veeam backup job? As it should be depricated if I am right.

Userlevel 7
Badge +12

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

Userlevel 7
Badge +7

@Link State Thank you for your graphical explenation 😃

Can you explain to me why you are backing up also by Veeam backup job? As it should be depricated if I am right.

You are right, but I prefer to make a unatantum backup, maybe I need some vmware conf\disk files and things like that. I prefer to have it :D
 

Userlevel 3
Badge

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

This makes it very clear. Better be safe then sorry. Thanks for that @regnor 

This is one of the errors:
3/2/2023 11:29:49 PM :: Processing [privatemachinename] Error: Failed to prepare VM for processing: Socket has been forcibly closed. Failed to recover connection.  
 

Userlevel 7
Badge +17

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

Exactly. This is the procedure we do,too.

And as said before, I like the Veeam backup to have all of the vCenter VM in the backup. Just in case...

Userlevel 7
Badge +7

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

This makes it very clear. Better be safe then sorry. Thanks for that @regnor 

This is one of the errors:
3/2/2023 11:29:49 PM :: Processing [privatemachinename] Error: Failed to prepare VM for processing: Socket has been forcibly closed. Failed to recover connection.  
 

try queuing the vcsa as the last job,like this example

or try upgrade vmware tools & migrate Vm su another datastore.

 

@regnor agree ;) fast & furious restore :P

Userlevel 3
Badge

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

This makes it very clear. Better be safe then sorry. Thanks for that @regnor 

This is one of the errors:
3/2/2023 11:29:49 PM :: Processing [privatemachinename] Error: Failed to prepare VM for processing: Socket has been forcibly closed. Failed to recover connection.  
 

try queuing the vcsa as the last job,like this example

or try upgrade vmware tools & migrate Vm su another datastore.

 

@regnor agree 😉 fast & furious restore :P

 

I rescheduled the vcsa job and let’s see what will be the result. Thanks for the help everyone. 

Userlevel 7
Badge +7

Lovely discussion!

Since vsphere 5.5 (If I don't remember bad) I rely the center backup to veeam, 

backup and replica.

also using embebed backup option, but when having issues or “ransom-problems” 
I always restored from Veeam backups or replicas without any issue!

We always backed up vCenter in a separated task, just to keep it schedule different from any other task, also the replica.

cheers-.

Userlevel 7
Badge +12

@ralphrenierkens Looking forward to your results. The error could have many causes, so maybe vCenter itself isn't the problem.

Comment