VMware vCenter Appliance: image backup is deprecated since 7.0

vNote42
Userlevel 7
Badge +13

I had the discussion two times last week, so I think it is worth sharing here.

You may have noticed, VMware deprecated image level backup of its vCenter Appliance (VCSA) with vSphere 7. What does this mean? Basically, it is still supported to do image level backup and restore with VCSA 7.0. But it will not be supported in future releases. Notice that U1 and U2 are still 7.0 versions. With 7.n, with n>0, VMware will not support image restore of VCSA any more. 

From my perspective, this step was taken because problems arise when restoring this Linux appliance. I have seen many appliances requiring a filesystem check after restore or crash. When you are lucky, everything works fine afterwards. If not, you have to file a support ticket.

To avoid this, VMware offers a new way for VCSA backup. Since 6.5 it is possible to run file-based backups and restore of VCSA. Since 6.7 this can be scheduled. With this, new protocols are supported as well: FTP, FTPS, HTTP, HTTPS, SFTP, NFS, or SMB. Please correct me, if I am wrong: with vSphere 7 (or U1?) SMB Version greater 1 is supportet since.

https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vcenter.install.doc/GUID-1C73996F-8312-4BBD-A16C-B2C8FC3C0D31.html

So if you did not set up file-base VCSA backup, do so now:

  1. Connect to your VCSA on port 5480 using a browser.
    Using root or SSO-admin account
  2. Click Backup → Configure

     

  3. Set up Backup Schedule

 

To test it, you can click Backup now and enable Use backup location and user name from backup schedule

 

Wolfgang | vnote42.net | @vNote42

26 comments

ralphrenierkens
Userlevel 3
Badge

@ralphrenierkensLooking forward to your results. The error could have many causes, so maybe vCenter itself isn't the problem.

 

Putting vcsa in a different job and schedule after the latest job run did do the trick. There are still some machines that have some issues but it has a different reason that we know about. So for now, you guys helped me a lot and I got much wiser 😀.

Have a great weekend you all!

regnor
Userlevel 7
Badge +14

@ralphrenierkens Looking forward to your results. The error could have many causes, so maybe vCenter itself isn't the problem.

Max M. | Systems Engineer @Veeam | VMCE/VMCA | VeeaMVP 2023 | former Legend & Vanguard
HunterLAFR
Userlevel 7
Badge +8

Lovely discussion!

Since vsphere 5.5 (If I don't remember bad) I rely the center backup to veeam, 

backup and replica.

also using embebed backup option, but when having issues or “ransom-problems” 
I always restored from Veeam backups or replicas without any issue!

We always backed up vCenter in a separated task, just to keep it schedule different from any other task, also the replica.

cheers-.

Luis F.- HunterLF - HunterLAFR - lfconsulting.org
ralphrenierkens
Userlevel 3
Badge

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

This makes it very clear. Better be safe then sorry. Thanks for that @regnor 

This is one of the errors:
3/2/2023 11:29:49 PM :: Processing [privatemachinename] Error: Failed to prepare VM for processing: Socket has been forcibly closed. Failed to recover connection.  
 

try queuing the vcsa as the last job,like this example

or try upgrade vmware tools & migrate Vm su another datastore.

 

@regnor agree 😉 fast & furious restore :P

 

I rescheduled the vcsa job and let’s see what will be the result. Thanks for the help everyone. 

Link State
Userlevel 7
Badge +10

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

This makes it very clear. Better be safe then sorry. Thanks for that @regnor 

This is one of the errors:
3/2/2023 11:29:49 PM :: Processing [privatemachinename] Error: Failed to prepare VM for processing: Socket has been forcibly closed. Failed to recover connection.  
 

try queuing the vcsa as the last job,like this example

or try upgrade vmware tools & migrate Vm su another datastore.

 

@regnor agree ;) fast & furious restore :P

Veeam: VMCA | VMCE | VMXP | Veeam Legend - Microsoft: MCITP | MCP | MCSA | 2008 R2 | 2012R2 | 2016 | MCSE Core Infrastructure | MCSE Cloud Platform - Azure: AZ900 | AZ104 - VMWare: VCP-DCV Vsphere 7.x - Cisco: CCNA (Expired)
JMeixner
Userlevel 7
Badge +17

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

Exactly. This is the procedure we do,too.

And as said before, I like the Veeam backup to have all of the vCenter VM in the backup. Just in case...

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
ralphrenierkens
Userlevel 3
Badge

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

This makes it very clear. Better be safe then sorry. Thanks for that @regnor 

This is one of the errors:
3/2/2023 11:29:49 PM :: Processing [privatemachinename] Error: Failed to prepare VM for processing: Socket has been forcibly closed. Failed to recover connection.  
 

Link State
Userlevel 7
Badge +10

@Link State Thank you for your graphical explenation 😃

Can you explain to me why you are backing up also by Veeam backup job? As it should be depricated if I am right.

You are right, but I prefer to make a unatantum backup, maybe I need some vmware conf\disk files and things like that. I prefer to have it :D
 

Veeam: VMCA | VMCE | VMXP | Veeam Legend - Microsoft: MCITP | MCP | MCSA | 2008 R2 | 2012R2 | 2016 | MCSE Core Infrastructure | MCSE Cloud Platform - Azure: AZ900 | AZ104 - VMWare: VCP-DCV Vsphere 7.x - Cisco: CCNA (Expired)
regnor
Userlevel 7
Badge +14

We also do it like @Link State. The image backup is faster to restore then the VCSA integrated backup. The later one is just the fallback if the image restore fails.

And normally vCenter is backed up together with other VMs, so no seperate job. What errors to you see at the 12AM job?

 

Max M. | Systems Engineer @Veeam | VMCE/VMCA | VeeaMVP 2023 | former Legend & Vanguard
ralphrenierkens
Userlevel 3
Badge

@Link State Thank you for your graphical explenation 😃

Can you explain to me why you are backing up also by Veeam backup job? As it should be depricated if I am right.

Link State
Userlevel 7
Badge +10

@JMeixner: While it is true it does not save any files “outside the VM” such as .vmx files you don’t need those to restore the vCenter Server from the file-based backup regardless. Could an Instant VM Recovery bring the vCenter back faster? Yes, absolutely, but I guess we just need to adjust to change :grinning:

During a disaster it happened that the VCSA restore was successfully completed, but the VCSA was misaligned because the old restore was used. We had to extract the ESxi from the cluster and re-insert it to synchronise everything.

Veeam: VMCA | VMCE | VMXP | Veeam Legend - Microsoft: MCITP | MCP | MCSA | 2008 R2 | 2012R2 | 2016 | MCSE Core Infrastructure | MCSE Cloud Platform - Azure: AZ900 | AZ104 - VMWare: VCP-DCV Vsphere 7.x - Cisco: CCNA (Expired)
Link State
Userlevel 7
Badge +10

i backup the VM VCSA 7 with veeam job, in combo with native VCSA backup, it is convenient because it supports several protocols. Uses nfs which can also be exposed on Windows

Veeam: VMCA | VMCE | VMXP | Veeam Legend - Microsoft: MCITP | MCP | MCSA | 2008 R2 | 2012R2 | 2016 | MCSE Core Infrastructure | MCSE Cloud Platform - Azure: AZ900 | AZ104 - VMWare: VCP-DCV Vsphere 7.x - Cisco: CCNA (Expired)
ralphrenierkens
Userlevel 3
Badge

Interesting topic. We are facing this issue as well at one of our clients. I would like to have good arguments to make some changes in the backup configurations. 

The question is not IF we need a backup but how to do it as efficient as possible. keep in mind that the backup is worthless if we can not restore. 😉

Configuration atm is as follows:
4 default vm backups based on vmware tags and with storage integration. Backup jobs are running every evening between 10 PM and 4 AM.

For some reason one backup job fails around 12 Am and I think that at that particular moment vcenter vm is creating a snapshot and the rest of the vm’s are failing within this job. Because of this reason we would like to create a new job with vcenter VM only.

is this the way to go or would wou guys suggest to do it differently?

vNote42
Userlevel 7
Badge +13

Good to know! However I think is still easier do VCSA backup with Veeam B&R.

Yes, backup is easier! But keep in restore in mind too!

Wolfgang | vnote42.net | @vNote42
wolff.mateus
Userlevel 7
Badge +11

Good to know! However I think is still easier do VCSA backup with Veeam B&R.

IT Lover | Veeam Vanguard | VUG Leader | Blogger
regnor
Userlevel 7
Badge +14

[update]

I forgot to mention, that it is just possible to restore these file-base restore points, when you have exactly the same vCenter Version (incl. build) as ISO file.

Therefore I recommend to download full VCSA ISO - with same version ofupdate/upgrade - afterwards.


And archive those ISOs on a safe place so that you can easily find them if needed. I've never had the problem with VMware but I often had to search for older Microsoft Exchange CUs and Microsoft doesn't offer them for download anymore...😅

Max M. | Systems Engineer @Veeam | VMCE/VMCA | VeeaMVP 2023 | former Legend & Vanguard
Shaokat
Userlevel 7
Badge +3

[update]

I forgot to mention, that it is just possible to restore these file-base restore points, when you have exactly the same vCenter Version (incl. build) as ISO file.

Therefore I recommend to download full VCSA ISO - with same version ofupdate/upgrade - afterwards.

thanks for your update 

Shaokat
JMeixner
Userlevel 7
Badge +17

Yes, VSAN has lost reference to a whole bunch of files. Among others were the VMX and VMXF files of the VCSA.

 

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
vNote42
Userlevel 7
Badge +13

[update]

I forgot to mention, that it is just possible to restore these file-base restore points, when you have exactly the same vCenter Version (incl. build) as ISO file.

Therefore I recommend to download full VCSA ISO - with same version ofupdate/upgrade - afterwards.

Wolfgang | vnote42.net | @vNote42
Shaokat
Userlevel 7
Badge +3

i agree with you.

Shaokat
vNote42
Userlevel 7
Badge +13

VSAN had lost references to some files - the configuration files of the VCSA were among them. The files were there but VSAN did not find them. The VMDK files of the VCSA were there but it was not startable because of the missing VMX Files and so on. Restore of these files would have been enough to get the VCSA working again. We have tried this with several other VMs.

As I said, it was a special case… The VMWare admin colleagues are in discussion with VMWare about this issue...

I am afraid I do not understand your problem completely too. Was your problem that you have lost the VMX-file of your VCSA?

Wolfgang | vnote42.net | @vNote42
JMeixner
Userlevel 7
Badge +17

VSAN had lost references to some files - the configuration files of the VCSA were among them. The files were there but VSAN did not find them. The VMDK files of the VCSA were there but it was not startable because of the missing VMX Files and so on. Restore of these files would have been enough to get the VCSA working again. We have tried this with several other VMs.

As I said, it was a special case… The VMWare admin colleagues are in discussion with VMWare about this issue...

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
haslund
Userlevel 7
Badge +14

@JMeixner: I’d love to better understand what the issue was here. Are you saying because it was VCSA on VSAN it was not possible to restore the VCSA using the file-based backup?

Check out my blog on https://rasmushaslund.com - Rasmus is a Veeam Certified Trainer and passionate about all community work.
JMeixner
Userlevel 7
Badge +17

Yes, I agree. And my case was rather special,  I guess.

We are doing the file-based backup. It is the supported method with V7.1 and later anyway.

But it is no big problem for us to do the snapshot backup additionally to have the configuration files at hand. This issue was not a VCSA problem, but a VSAN problem.

 

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
haslund
Userlevel 7
Badge +14

@JMeixner: While it is true it does not save any files “outside the VM” such as .vmx files you don’t need those to restore the vCenter Server from the file-based backup regardless. Could an Instant VM Recovery bring the vCenter back faster? Yes, absolutely, but I guess we just need to adjust to change :grinning:

Check out my blog on https://rasmushaslund.com - Rasmus is a Veeam Certified Trainer and passionate about all community work.

Comment


Terms & Conditions