Skip to main content
  • EN
Question

vCLS appliance showed up in one of our backup jobs. This caused the backup job to show as failed.

vCLS appliance showed up in one of our backup jobs. This caused the backup job to show as failed.

To workaround the issue, we excluded the vCLS appliance from the job. For some reason, this should have been in the vCLS folder but it appeared in a the Discovered Virtual Machine folder in vCenter. There are other vCLS appliances in the same volume but since they are all in the vCLS folder, Veeam does not back those up.

Our vSphere admins looked at the issue and not sure what is going on in here. I think this should be a question to the vmware experts but wondering if any of you guys have seen this behavior. What do you think is going on in here?

 

16 comments

coolsport00
Userlevel 7
Badge +19

Interesting....never heard of this happening before @spider32 

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
Dynamic
Userlevel 6
Badge +6

strange issue, a normal running vCLS system should never be part of the backup.

Is it possible that you recently had discovered a problem in your vSphere Cluster? Modified HA settings, added/removed hosts? Or maybe changed settings regarding the retread mode (where you can also deprovision the vCLS systems and also deploy them from scratch)? 

 

edit: If so, then *this* vCLS machine could potentially be an orphaned vCLS system - you should be able to delete this (option not greyed out) in the context menu. But if this is true, your vSphere team, should also check, what causes this behaviour.

 

Best regards Markus

Markus Hartmann | Veeam Legend 2024 | VMCA 2024 & VMCE 2024 | VMware Certified Implementation Expert - Data Center Virtualization 2024 | https://blog.4it.gmbh/
coolsport00
Userlevel 7
Badge +19

Hi @spider32 , I was searching around the Hub here to see if I could find more info related to vCLS VMs being processed and came across the following post:

It appears since Veeam v10a, Veeam automatically excludes vCLS VMs from being processed/backed up, although I couldn’t find definitive documentation stating such (maybe Release Notes for that version no longer Internet accessible).

How do you configure VMs to be backed up in your Jobs?...by Datastore container maybe? If so, maybe Veeam somehow backed up a vCLS VM in one of your Datastores you use as a container to backup from? You can control what Datastore vCLS VMs get placed on...maybe try this as well?

https://docs.vmware.com/en/VMware-vSphere/7.0/com.vmware.vsphere.resmgmt.doc/GUID-6C11D7F9-4E92-4EA8-AA63-AABAD4B299E7.html

Still, if Veeam does have automatic exclusions for vCLS VMs (and makes sense to me they should), you should probably ping Veeam Support to see why, at least for this 1 Job of yours and this one time, Veeam did backup a vCLS VM. I do assume you’re on a Veeam version later than 10a?

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
Andanet
Userlevel 7
Badge +10

A few years ago I has a similar issue (not backup related) and I remember my action was to disable and enable the vcls VM. 

Antonio D'Andrea | VMCE 2021 | VMCA 2022 | VMTSP2024 | Veeam Legend 2023/2024 | VUG Italy Leader | VCP6-DCV | I want it written on my tombstone "Please start a restore to the last valid backup"
S
Userlevel 6
Badge +1

Our backup jobs are volume based. So, it backups all VMs in a particular volume. It so happened that vSphere decided to put all the vCLS appliances in this volume which is totally fine. Like what @coolsport00 mentioned, Veeam automatically excluded all the vCLS appliances from the backup except this one. I think Veeam does the exclusion by using the vCLS folder. However, this particular vCLS appliance for some weird reason is not in the vCLS folder and we are unable to move this to the vCLS folder thus the backup job tries to backup this appliance. Workaround was manual exclusion but really curious how this happened and what would be the fix. Powering off/on the appliance is not working. We tried to move the vCLS appliance to the vCLS folder but getting Operation failed permission denied (as a vCenter admin). I have relayed the added info to our vSphere admin. Let you know any updates. Thanks.

Dynamic
Userlevel 6
Badge +6

hi @spider32, as mentioned, check retreat mode in vSphere. Your vSphere Admin could simply deprovision these vCLS systems with a single click, and automatically redeploy them.
There must something happened in your vSphere Cluster regarding the vCLS to get orphaned/corrupted or something like this. 
best regards Markus

Markus Hartmann | Veeam Legend 2024 | VMCA 2024 & VMCE 2024 | VMware Certified Implementation Expert - Data Center Virtualization 2024 | https://blog.4it.gmbh/
S
Userlevel 6
Badge +1

Yes, retreat mode, @Dynamic,  was suggested by our vsphere admin but since this is a production environment and the issue is not really critical, they would like to schedule when to do this one. I will update if this resolves the issue. Thanks.

coolsport00
Userlevel 7
Badge +19

@spider32 - ok..keep us posted how things go.

Best.

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
B

This began happening to me recently after I upgraded vCenter/VCSA and Veeam to the latest version of each. VCSA 8.0.3.0000 and Veeam 12.1.2.172. I have excluded the offending VCLs VMs but I’m not sure why they’re being included in my backup jobs at all. I forced VCSA to recreate the VMs but that hasn’t resolved the problem.

coolsport00
Userlevel 7
Badge +19

@bbrown57 - I recommend reaching out to Veeam Support to see why Veeam isn’t excluding those vCLS VMs by default.

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
Dynamic
Userlevel 6
Badge +6

Hi @bbrown57, i agree with @coolsport00 . But be aware that 8.0U3 is currently not supported by Veeam (KB2443), even though it’s working. I think you will see also a warning within your jobs.

But your problem is indeed usefull for the support team, because it’s maybe related to U3.
Regarding U3, Veeam will offer support normally within 90 days since GA.

Greeting Markus

Markus Hartmann | Veeam Legend 2024 | VMCA 2024 & VMCE 2024 | VMware Certified Implementation Expert - Data Center Virtualization 2024 | https://blog.4it.gmbh/
Dynamic
Userlevel 6
Badge +6

Hi @bbrown57 - did you reach out to the support?

...i can confirm that this is an issue regarding 8U3 (which as mentioned is not yet supported). A colleague of me, is running in the exact same issue.

So you have to wait for the next VBR, when 8U3 will be officially supported, then imo this would be adressed. In the meantime you have to exclude these vCLS systems.

Markus Hartmann | Veeam Legend 2024 | VMCA 2024 & VMCE 2024 | VMware Certified Implementation Expert - Data Center Virtualization 2024 | https://blog.4it.gmbh/
B

@Dynamic Yes, I reached out to support. They also confirmed to me that it is a vCenter 8U3 issue. They said they should have a patch in August to resolve the problem. For now I just excluded the VCL VMs from the backup job and it’s been working fine.

Dynamic
Userlevel 6
Badge +6

@Dynamic Yes, I reached out to support. They also confirmed to me that it is a vCenter 8U3 issue. They said they should have a patch in August to resolve the problem. For now I just excluded the VCL VMs from the backup job and it’s been working fine.

glad to hear :)

Also you could exclude the whole vCLS folder (or work with a vSphere Tag like “nobackup” and those tag to the folder and define an exclusion within VBR for this tag).

 

Markus Hartmann | Veeam Legend 2024 | VMCA 2024 & VMCE 2024 | VMware Certified Implementation Expert - Data Center Virtualization 2024 | https://blog.4it.gmbh/
S
Userlevel 6
Badge +1

We are still on 7.0.3. We are still working on scheduling doing the Retreat mode. It’s currently low priority for the Virtualization Team due to the workaround. 

I will open a ticket to support since we are in a holding pattern due to the Retreat mode implementation.

 

coolsport00
Userlevel 7
Badge +19

We are still on 7.0.3. We are still working on scheduling doing the Retreat mode. It’s currently low priority for the Virtualization Team due to the workaround. 

I will open a ticket to support since we are in a holding pattern due to the Retreat mode implementation.

 

Thanks for the update @spider32 

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00

Comment


Terms & Conditions