Solved

Backup & Replication console won't connect after upgrade to 12.1.0.2131


Userlevel 3

Hi,  We’ve just upgraded our Veeam Backup and Replication server to v12.1.0.213.  All went very smoothly.  I also upgraded the Backup and Replication Console, which sits on another server, to the same version. We usually use this for access to Veeam.  Again, the upgrade went well with nothing untoward being apparent.  

However, now my Veeam console will not log in to the Veeam Server. I get a message saying “Failed to connect VeeamBackup & Replication server: The logon attempt failed.”

On investigation, I found that this occurred when I was using the server name (as the field was auto-populated with this info from before the upgrade).  However, if I use the IP of the Veeam server, the console connects as expected.  I tested this using the full FQDN of the Veeam server, but still did not connect - it only seems to connect using the IP. 

Has anyone got any suggestions of how I can connect using server name again?

Thanks. 

icon

Best answer by phil123456789 29 January 2024, 12:48

View original

34 comments

Userlevel 1

I so hoped this will be solved in 12.1.1.56 :-(

Wow really not fixed. 20yr enterprise customer here saying “shame”.

I so hoped this will be solved in 12.1.1.56 :-(

Userlevel 7
Badge +3

Glad to hear you issue is  resolved 

Userlevel 7
Badge +20

100% a Veeam upgrade issue, no changes here pre-upgrade and I'm having this issue too.

Every VBRC has this across all of our clients that we manage via VSPC.  

One we notice, is regardless if you upgrade via VSPC or local, it installs a local server backup and causes VSPC to fail loading.

Now, once we spent 24 hours fixing this garbage, none of our servers will connect to vSPC.  No cert issues.  

Yeah, I’m beyond pissed, and the fact I can’t just call someone for support, and nothing is updated in forums about all these issues. 

Yeah,  maybe you can tell I’m pissed off.

This is not the place to take this attitude or language. If you have a complaint then post it on the forums as there is no one here from the Veeam Product management team!

100% a Veeam upgrade issue, no changes here pre-upgrade and I'm having this issue too.

Every VBRC has this across all of our clients that we manage via VSPC.  

One we notice, is regardless if you upgrade via VSPC or local, it installs a local server backup and causes VSPC to fail loading.

Now, once we spent 24 hours fixing this garbage, none of our servers will connect to vSPC.  No cert issues.  

Yeah, I’m beyond pissed, and the fact I can’t just call someone for support, and nothing is updated in forums about all these issues. 

Yeah,  maybe you can tell I’m pissed off.

Userlevel 3

Had this problem immediately, and can reproduce on fresh installs. Un joining and rejoing a backup server in a production environment as a solution is problematic to say the least.

I agree.  At the moment I’m living with it as-is and had to reconnect my handful of desktop clients to point by IP instead of FQDN.  I’ll probably end up doing a fresh install and migrate the server off the domain per best practices, but that will take some time to plan out properly.

Userlevel 1

Had this problem immediately, and can reproduce on fresh installs. Un joining and rejoing a backup server in a production environment as a solution is problematic to say the least.

Userlevel 7
Badge +20

Glad to hear you resolved the issue as well as sharing what the solution was.  It is too bad you had to rejoin the domain again.

Userlevel 7
Badge +17

Nice! Glad to hear you got it sorted, & thanks for providing an update. 

Userlevel 3

SOLVED

Hi, just to update on my particular case.  

We tried the remove and re-add to the Domain this morning.  This seems to have solved the issue.  I can now log in via the console using the server name, as well as the IP. 

Thanks to everyone for the advice on this issue. Much appreciated.  💪

Userlevel 1

100% a Veeam upgrade issue, no changes here pre-upgrade and I'm having this issue too.

worth mentioning that i said “no changes here pre-upgrade” when there was the enterprise manager upgrade prior on the same server.

Userlevel 1

100% a Veeam upgrade issue, no changes here pre-upgrade and I'm having this issue too.

Userlevel 7
Badge +17

@sosinfo1767 -

Yeah..curious what Support finds. Let us know what the solution is when you can.

Userlevel 7
Badge +20

This happened to us aswell. only change in our enviroment was that we upgraded to latest service pack for Veeam.   I have a feeling that in the upgrade there is a FQDN issue.  I am going to open a call.  I forsee a hotfix to fix the issue

 

Here is hoping the fix is easy as they already released CP1 - 12.1.1.56.  😂

Hope those affected don’t have to wait until CP2.

I am on 10.1.1.56,  that is the update that caused my issue.  

Oh darn.  😔

Well hopefully Support can get it worked out or fixed.

Userlevel 1

This happened to us aswell. only change in our enviroment was that we upgraded to latest service pack for Veeam.   I have a feeling that in the upgrade there is a FQDN issue.  I am going to open a call.  I forsee a hotfix to fix the issue

 

Here is hoping the fix is easy as they already released CP1 - 12.1.1.56.  😂

Hope those affected don’t have to wait until CP2.

I am on 10.1.1.56,  that is the update that caused my issue.  

Userlevel 7
Badge +20

This happened to us aswell. only change in our enviroment was that we upgraded to latest service pack for Veeam.   I have a feeling that in the upgrade there is a FQDN issue.  I am going to open a call.  I forsee a hotfix to fix the issue

 

Here is hoping the fix is easy as they already released CP1 - 12.1.1.56.  😂

Hope those affected don’t have to wait until CP2.

Userlevel 1

This happened to us aswell. only change in our enviroment was that we upgraded to latest service pack for Veeam.   I have a feeling that in the upgrade there is a FQDN issue.  I am going to open a call.  I forsee a hotfix to fix the issue

 

Userlevel 3

we are small, so I’m not really concerned about workgroups in a practical sense…  I’ll be the only one with the login credentials with a “break glass” access for one other person in case I’m “hit by a bus”.  What is an issue with workgroups is that it definitely puts a hiccup in our eventual CMMC 2.0 lvl2 aspirations. 

Userlevel 6
Badge +3

@NZ_BenThomas Thanks for that.  I had come across that in the past and some of the weaknesses of the workgroup only held me off, but we are a small shop and doing a management domain isn’t much better than a single domain because… what is backing that up and running that when everything is dead?  I started looking at figuring out if a one-way trust between on-site AD and Azure is possible so we can use MS Entra for the identity management for the management devices.

In small scenarios, workgroup makes sense. I’d say if you’ve got a password management solution, see if it’s got an ability to rotate local credentials for you, so that you can change the local user password at a regular interval automatically, or see if it’s got an API for you to script that.

The main issue with local users on workgroup is shared accounts with passwords that don’t change often.

Userlevel 3

@NZ_BenThomas Thanks for that.  I had come across that in the past and some of the weaknesses of the workgroup only held me off, but we are a small shop and doing a management domain isn’t much better than a single domain because… what is backing that up and running that when everything is dead?  I started looking at figuring out if a one-way trust between on-site AD and Azure is possible so we can use MS Entra for the identity management for the management devices.

Userlevel 6
Badge +3

@phil123456789 I had the same exact issue you had.  I would also add that this extends to unmanaged backup computers also failing to connect to the BR server as a repository using the FQDN while using the IP works.  DNS here is also good and there were no major changes on anything other than upgrading BR from 12.0 (latest patch) to 12.1.  I had too many other items open and support wasn’t really helpful at all in even considering that the issue could possibly exist insisting that it must be DNS (yes, I know “its always DNS”).  Anyway… my case number was 07061577, but I let them close it because I just didn’t have the personal bandwidth to fight them on this when IP worked.  the BR server is domain joined and I'm seriously thinking about how to move the BR infrastructure out of AD and use fixed IP only so it is a little less of a chicken/egg situation during a full lights-out DR situation.

Some good info on the pros/cons of workgroup vs domain for VBR

Workgroup or Domain ? - Veeam Backup & Replication Security Best Practice Guide

Userlevel 3

@phil123456789 I had the same exact issue you had.  I would also add that this extends to unmanaged backup computers also failing to connect to the BR server as a repository using the FQDN while using the IP works.  DNS here is also good and there were no major changes on anything other than upgrading BR from 12.0 (latest patch) to 12.1.  I had too many other items open and support wasn’t really helpful at all in even considering that the issue could possibly exist insisting that it must be DNS (yes, I know “its always DNS”).  Anyway… my case number was 07061577, but I let them close it because I just didn’t have the personal bandwidth to fight them on this when IP worked.  the BR server is domain joined and I'm seriously thinking about how to move the BR infrastructure out of AD and use fixed IP only so it is a little less of a chicken/egg situation during a full lights-out DR situation.

Userlevel 6
Badge +3

Also just to confirm, when you're using the consope are you logging in as a domain user or a local user?

Userlevel 6
Badge +3

Hi,  DNS seems to work forward and reverse. 

We are using Kerberos, and I think it might have something to do with those SPNs.  I found an error in the Windows System event logs on the server where the console is installed:

Event ID: 4

“The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server <service account>. The target name used was VeeamBackupSvc/<backup svr hostname>.  this indicates the the target server failed to decrypt the ticket provided by the client.  This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using.  Ensure that the target SPN is only registered on the account used by the server.  This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service.  Ensure that the service on the server and the KDC are both configured to use the same password.  If the server name is not fully qualified and the target domain <domain> is different from the client domain <same domain>, check if there are identically named server accounts in these two domains, or use the fully qualified name to identify the server.” 

I’m kind of at the limit of my Windows AD knowledge here, I dabble when necessary - but its not usually my bag.

Thanks - 

Have a look through this section of the docs which explains what's needed for Kerberos authentication.

https://helpcenter.veeam.com/docs/backup/vsphere/kerberos_authentication.html?ver=120

The SPNs should register when the services start, so you could try restarting the Veeam server.

Otherwise you can manually configure the SPNs following this https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/register-a-service-principal-name-for-kerberos-connections?view=sql-server-ver16#Manual

Userlevel 7
Badge +20

Is this VBR server part of a domain?  Maybe rejoining it to the domain could possibly fix it.  Did anything else other than upgrading Veeam happen on the domain?

Comment