Veeam EM v12 Upgrade - Antivirus Problem


Userlevel 7
Badge +7

Hi all,

I want to share with you a problem I encountered on a customer while upgrading Veeam Enterprise Manager from v11a to v12 latest release.

 

The environment

 

This is a simple Veeam v11a installation, VBR + EM.

The customer uses Windows Defender for Endpoint (Antivirus/EDR) on the server, the exclusions are correctly set as per KB https://www.veeam.com/kb1999.

 

The upgrade..Ouch!

 

After successfully updating a dozen Veeam environments to v12, I start this new upgrade with the confidence "It works!".

Once I get to the end of the wizard and start the update, at some point you receive the error "Failed to create website 0x80070020".

This error is reported in the KB https://www.veeam.com/kb1992, where it is actually recommended to disable the antivirus.

 

Support case and solution

 

I was followed by a good Veeam support engineer, but there was nothing to do..unfortunately it was not enough to disable the AV (we tried everything, registry, powershell, voodoo rituals), but it was necessary to completely remove it from the server to complete the upgrade successfully.

 

Conclusion and advice

 

I was sorry that a clean solution was not found, the customer is new to Veeam and this workaround was not good!

Ps: always remember to take a snapshot of the server before upgrading..the above error completely destroys the EM installation!

💚


7 comments

Userlevel 7
Badge +7

Thanks for the heads up! 

Sounds like it might have been Windows Defender ATP with Tamper Protection enabled which is why it might have given so much difficulty in trying to uninstall it. 

A way round would be to offboard and the disable Windows Defender.

Userlevel 7
Badge +13

Mmm in my experience @marco_s if AV is correctly disabled it’s impossible that could be the cause of that error. But hey, I have zero experience with WInDefender.

That’s seems to be more likely related to a IIS issue (and Veeam Enterprise Manager has some IIS entries as you can see here: https://www.veeam.com/kb1168)

Did you check when you got the error which processes used 80 and 443?

netstat -aon | find ":80" 
netstat -aon | find ":443"

 

Userlevel 7
Badge +7

Hi @dips and @marcofabbri ..we were able to disable tamper protection, real-time protection and all other active services, but Windows Defender ATP Service was still running! 🙄

@marcofabbri we checked IIS config with Veeam support and all was fine 🙁

Userlevel 7
Badge +20

Unusual problem for sure.  Hopefully there is a solution that comes out of this.

Same experience as OP. 
 "Failed to create website 0x80070020".
Simple standalone Windows Server 2016 (NON-DOMAIN) (VBR + EM)

EM upgrade is hit with this error and a fatal message. Then install fails and I have no EM installation any more. Enterprise Manager service is gone.

In 10+ years of installs and upgrades this is a first.

As mentioned above, I have checked and only have Windows Defender installed.

Realtime Protection is on and it appears with Tamper Protection enabled. (I shouldn’t have had this running 🤦‍♂️) - where was the warning from the setup wizard!
I have attempted to disable via Local Group Policy this Realtime Protection.
It shows disabled within Windows Security but within Server Manager its enabled!

I visit Windows Features and see the Windows Defender listed but there is no ability to remove - its greyed out. (I am local administrator!)

There is no other third party AV or malware or protection on the server.

How the HECK do you disable? Whats this about malware protection from Veeam when you have to disable or remove this stuff on the VBR server to upgrade their software, then enable it again.

I even have the exclusions setup also.

But no longer have Enterprise Manager. I think I have a half baked VeeamBackup entity in IIS.


The setup wizard recognises I have no EM. But prompts to want to upgrade VBR.

Which I am not touching in case it BORKS that also.

Confidence at all time low. 👎
I upgraded Veeam on other customers server without hitch last week. (Granted it did not have EM on it however)

For those that like nice pictures.

 

Any advice would be great.

Call support, seems pointless from poster above, as onus is on yourself to disable any AV interaction.

I do see now some mention of this error in the Upgrade checklist on the helpcenter specifically for EM. But i cannot even disable baked in services to the Windows OS.

See you on the other side hopefully.

Userlevel 7
Badge +8

That snapshot advice is so important. It’s easy to say “Oh, this is an easy upgrade, it’ll just work”.

Those are the times where things go bad in a hurry.

I just had this happen on a vSphere patch and didn’t snapshot the vCenter server 🤣. Lucky it was a lab environment. Now, even the lab environment gets snapshots before upgrades!!

I got EM to install when upgrading from v11 to 12 essentially by disabling UAC. Details at Enterprise Manager: Failed to create website 0x80070020 - R&D Forums (veeam.com).

Comment