• EN
Solved

Backup & Replication console won't connect after upgrade to 12.1.0.2131

P
Userlevel 3

Hi,  We’ve just upgraded our Veeam Backup and Replication server to v12.1.0.213.  All went very smoothly.  I also upgraded the Backup and Replication Console, which sits on another server, to the same version. We usually use this for access to Veeam.  Again, the upgrade went well with nothing untoward being apparent.  

However, now my Veeam console will not log in to the Veeam Server. I get a message saying “Failed to connect VeeamBackup & Replication server: The logon attempt failed.”

On investigation, I found that this occurred when I was using the server name (as the field was auto-populated with this info from before the upgrade).  However, if I use the IP of the Veeam server, the console connects as expected.  I tested this using the full FQDN of the Veeam server, but still did not connect - it only seems to connect using the IP. 

Has anyone got any suggestions of how I can connect using server name again?

Thanks. 

icon

Best answer by phil123456789 29 January 2024, 12:48

View original

34 comments

MicoolPaul
Userlevel 7
Badge +20

If it’s working on IP but not FQDN it sounds like your DNS entry has gone wrong.

 

validate your dns record and if in doubt, temporarily bypass it with your hosts file. I’d do a packet capture next to see where the traffic is going and where that breaks down. Other than that it’s log diving time!

Michael Paul - #VeeamLegend | #VeeamVanguard | Veeam Certified Architect | VMware vExpert | https://micoolpaul.com | Twitter: @MicoolPaul | Mastodon: @micoolpaul@masto.nu | Bluesky: @micoolpaul.bsky.social
coolsport00
Userlevel 7
Badge +17

Yep...agree with Michael here 

Verify you can ping the VBR hostname from this server which runs the Console. Is this server on the domain? VBR on the domain? If not, you'll need to add backup environment severs to the hosts file as Michael suggests. 

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
Chris.Childerhose
Userlevel 7
Badge +20

 

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
P
Userlevel 3

Hi, I did check that and DNS seemed OK.  Both servers are in the same domain, and I can ping them from each other by server name.  I also tried the;

“Test-NetConnection -ComputerName <name> -Port 9392”

command on the server where the console is installed, and that came back successfully (where <name> is the Veeam server).  I’ll have a look in the logs and see if there is anything...

Chris.Childerhose
Userlevel 7
Badge +20

Hi, I did check that and DNS seemed OK.  Both servers are in the same domain, and I can ping them from each other by server name.  I also tried the;

“Test-NetConnection -ComputerName <name> -Port 9392”

command on the server where the console is installed, and that came back successfully (where <name> is the Veeam server).  I’ll have a look in the logs and see if there is anything...

Another thing to check is the services are running which I am sure you did but many of the Veeam services are the Delayed variety so maybe you tried to run the console right before they were fully running?   Other than that, logs should tell you more if DNS is working.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
coolsport00
Userlevel 7
Badge +17

Hmm..ok. Good to hear DNS is fine. Interesting you still can’t connect with hostname tho 🤔

Keep us posted what you find @phil123456789 

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
NZ_BenThomas
Userlevel 6
Badge +3

Have you secured the domain to enforce Kerberos rather than NTLM? If so, you might need to check that the correct SPNs are in place for the FQDN of the Veeam box.

Ben Thomas | Solutions Advisor | Veeam Vanguard 2023 | VMCE2022 | Microsoft MVP 2018-2023 | BCThomas.com
PeteSteven
Userlevel 4

Are the DNS entries forward and reverse resolvable?

https://bp.veeam.com/vbr/3_Build_structures/B_Other/dns_resolution.html

P
Userlevel 3

Hi,  DNS seems to work forward and reverse. 

We are using Kerberos, and I think it might have something to do with those SPNs.  I found an error in the Windows System event logs on the server where the console is installed:

Event ID: 4

“The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server <service account>. The target name used was VeeamBackupSvc/<backup svr hostname>.  this indicates the the target server failed to decrypt the ticket provided by the client.  This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using.  Ensure that the target SPN is only registered on the account used by the server.  This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service.  Ensure that the service on the server and the KDC are both configured to use the same password.  If the server name is not fully qualified and the target domain <domain> is different from the client domain <same domain>, check if there are identically named server accounts in these two domains, or use the fully qualified name to identify the server.” 

I’m kind of at the limit of my Windows AD knowledge here, I dabble when necessary - but its not usually my bag.

Thanks - 

Chris.Childerhose
Userlevel 7
Badge +20

Is this VBR server part of a domain?  Maybe rejoining it to the domain could possibly fix it.  Did anything else other than upgrading Veeam happen on the domain?

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
NZ_BenThomas
Userlevel 6
Badge +3

Hi,  DNS seems to work forward and reverse. 

We are using Kerberos, and I think it might have something to do with those SPNs.  I found an error in the Windows System event logs on the server where the console is installed:

Event ID: 4

“The Kerberos client received a KRB_AP_ERR_MODIFIED error from the server <service account>. The target name used was VeeamBackupSvc/<backup svr hostname>.  this indicates the the target server failed to decrypt the ticket provided by the client.  This can occur when the target server principal name (SPN) is registered on an account other than the account the target service is using.  Ensure that the target SPN is only registered on the account used by the server.  This error can also happen if the target service account password is different than what is configured on the Kerberos Key Distribution Center for that target service.  Ensure that the service on the server and the KDC are both configured to use the same password.  If the server name is not fully qualified and the target domain <domain> is different from the client domain <same domain>, check if there are identically named server accounts in these two domains, or use the fully qualified name to identify the server.” 

I’m kind of at the limit of my Windows AD knowledge here, I dabble when necessary - but its not usually my bag.

Thanks - 

Have a look through this section of the docs which explains what's needed for Kerberos authentication.

https://helpcenter.veeam.com/docs/backup/vsphere/kerberos_authentication.html?ver=120

The SPNs should register when the services start, so you could try restarting the Veeam server.

Otherwise you can manually configure the SPNs following this https://learn.microsoft.com/en-us/sql/database-engine/configure-windows/register-a-service-principal-name-for-kerberos-connections?view=sql-server-ver16#Manual

Ben Thomas | Solutions Advisor | Veeam Vanguard 2023 | VMCE2022 | Microsoft MVP 2018-2023 | BCThomas.com
NZ_BenThomas
Userlevel 6
Badge +3

Also just to confirm, when you're using the consope are you logging in as a domain user or a local user?

Ben Thomas | Solutions Advisor | Veeam Vanguard 2023 | VMCE2022 | Microsoft MVP 2018-2023 | BCThomas.com
A
Userlevel 3

@phil123456789 I had the same exact issue you had.  I would also add that this extends to unmanaged backup computers also failing to connect to the BR server as a repository using the FQDN while using the IP works.  DNS here is also good and there were no major changes on anything other than upgrading BR from 12.0 (latest patch) to 12.1.  I had too many other items open and support wasn’t really helpful at all in even considering that the issue could possibly exist insisting that it must be DNS (yes, I know “its always DNS”).  Anyway… my case number was 07061577, but I let them close it because I just didn’t have the personal bandwidth to fight them on this when IP worked.  the BR server is domain joined and I'm seriously thinking about how to move the BR infrastructure out of AD and use fixed IP only so it is a little less of a chicken/egg situation during a full lights-out DR situation.

NZ_BenThomas
Userlevel 6
Badge +3

@phil123456789 I had the same exact issue you had.  I would also add that this extends to unmanaged backup computers also failing to connect to the BR server as a repository using the FQDN while using the IP works.  DNS here is also good and there were no major changes on anything other than upgrading BR from 12.0 (latest patch) to 12.1.  I had too many other items open and support wasn’t really helpful at all in even considering that the issue could possibly exist insisting that it must be DNS (yes, I know “its always DNS”).  Anyway… my case number was 07061577, but I let them close it because I just didn’t have the personal bandwidth to fight them on this when IP worked.  the BR server is domain joined and I'm seriously thinking about how to move the BR infrastructure out of AD and use fixed IP only so it is a little less of a chicken/egg situation during a full lights-out DR situation.

Some good info on the pros/cons of workgroup vs domain for VBR

Workgroup or Domain ? - Veeam Backup & Replication Security Best Practice Guide

Ben Thomas | Solutions Advisor | Veeam Vanguard 2023 | VMCE2022 | Microsoft MVP 2018-2023 | BCThomas.com
A
Userlevel 3

@NZ_BenThomas Thanks for that.  I had come across that in the past and some of the weaknesses of the workgroup only held me off, but we are a small shop and doing a management domain isn’t much better than a single domain because… what is backing that up and running that when everything is dead?  I started looking at figuring out if a one-way trust between on-site AD and Azure is possible so we can use MS Entra for the identity management for the management devices.

NZ_BenThomas
Userlevel 6
Badge +3

@NZ_BenThomas Thanks for that.  I had come across that in the past and some of the weaknesses of the workgroup only held me off, but we are a small shop and doing a management domain isn’t much better than a single domain because… what is backing that up and running that when everything is dead?  I started looking at figuring out if a one-way trust between on-site AD and Azure is possible so we can use MS Entra for the identity management for the management devices.

In small scenarios, workgroup makes sense. I’d say if you’ve got a password management solution, see if it’s got an ability to rotate local credentials for you, so that you can change the local user password at a regular interval automatically, or see if it’s got an API for you to script that.

The main issue with local users on workgroup is shared accounts with passwords that don’t change often.

Ben Thomas | Solutions Advisor | Veeam Vanguard 2023 | VMCE2022 | Microsoft MVP 2018-2023 | BCThomas.com
A
Userlevel 3

we are small, so I’m not really concerned about workgroups in a practical sense…  I’ll be the only one with the login credentials with a “break glass” access for one other person in case I’m “hit by a bus”.  What is an issue with workgroups is that it definitely puts a hiccup in our eventual CMMC 2.0 lvl2 aspirations. 

S
Userlevel 1

This happened to us aswell. only change in our enviroment was that we upgraded to latest service pack for Veeam.   I have a feeling that in the upgrade there is a FQDN issue.  I am going to open a call.  I forsee a hotfix to fix the issue

 

Chris.Childerhose
Userlevel 7
Badge +20

This happened to us aswell. only change in our enviroment was that we upgraded to latest service pack for Veeam.   I have a feeling that in the upgrade there is a FQDN issue.  I am going to open a call.  I forsee a hotfix to fix the issue

 

Here is hoping the fix is easy as they already released CP1 - 12.1.1.56.  😂

Hope those affected don’t have to wait until CP2.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
S
Userlevel 1

This happened to us aswell. only change in our enviroment was that we upgraded to latest service pack for Veeam.   I have a feeling that in the upgrade there is a FQDN issue.  I am going to open a call.  I forsee a hotfix to fix the issue

 

Here is hoping the fix is easy as they already released CP1 - 12.1.1.56.  😂

Hope those affected don’t have to wait until CP2.

I am on 10.1.1.56,  that is the update that caused my issue.  

Chris.Childerhose
Userlevel 7
Badge +20

This happened to us aswell. only change in our enviroment was that we upgraded to latest service pack for Veeam.   I have a feeling that in the upgrade there is a FQDN issue.  I am going to open a call.  I forsee a hotfix to fix the issue

 

Here is hoping the fix is easy as they already released CP1 - 12.1.1.56.  😂

Hope those affected don’t have to wait until CP2.

I am on 10.1.1.56,  that is the update that caused my issue.  

Oh darn.  😔

Well hopefully Support can get it worked out or fixed.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
coolsport00
Userlevel 7
Badge +17

@sosinfo1767 -

Yeah..curious what Support finds. Let us know what the solution is when you can.

Shane Williford - Veeam VMCA/VMCE | Veeam Legend | VUG Leader | VCP-DCV | Twitter: @coolsport00
R
Userlevel 1

100% a Veeam upgrade issue, no changes here pre-upgrade and I'm having this issue too.

R
Userlevel 1

100% a Veeam upgrade issue, no changes here pre-upgrade and I'm having this issue too.

worth mentioning that i said “no changes here pre-upgrade” when there was the enterprise manager upgrade prior on the same server.

P
Userlevel 3

SOLVED

Hi, just to update on my particular case.  

We tried the remove and re-add to the Domain this morning.  This seems to have solved the issue.  I can now log in via the console using the server name, as well as the IP. 

Thanks to everyone for the advice on this issue. Much appreciated.  💪

Comment


Terms & Conditions