Skip to main content
Solved

Veeam vCenter backup - accessing incorrect IP address of VMware node


  • Not a newbie anymore
  • 7 comments

Being new to VEEAM, I might be troubleshooting problem that's trivial for VEEAM experts.

On Veeam 12.2 and vCenter 8.0.3, my problem is ESXi hosts having multiple networks which causes Veeam's confusion. There's management network and there's vMotion network. The hosts in vCenter are defined as follows:

The problem is, Veeam isn't able to back up a single VM. The log is full of these errors:

22.10.2024 11:17:51.481] <  1436> vdl      | [vddk] CnxOpenTCPSocket: Cannot connect to server 172.16.0.13:902: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond
[22.10.2024 11:17:51.481] <  1436> vdl      | [vddk] CnxAuthdConnect: Returning false because CnxAuthdConnectTCP failed
[22.10.2024 11:17:51.481] <  1436> vdl      | [vddk] CnxConnectAuthd: Returning false because CnxAuthdConnect failed
[22.10.2024 11:17:51.481] <  1436> vdl      | [vddk] Cnx_Connect: Returning false because CnxConnectAuthd failed
[22.10.2024 11:17:51.481] <  1436> vdl      | [vddk] Cnx_Connect: Error message: Failed to connect to server 172.16.0.13:902
[22.10.2024 11:17:51.481] <  1436> vdl      | WARN|[vddk] warn [NFC ERROR]NfcNewAuthdConnectionEx: Failed to connect: Failed to connect to server 172.16.0.13:902
[22.10.2024 11:17:51.481] <  1436> vdl      | WARN|[vddk] warn [NFC ERROR]NfcNewAuthdConnectionEx: Failed to connect to peer. Error: Failed to connect to server 172.16.0.13:902
[22.10.2024 11:17:51.481] <  1436> vdl      | WARN|[vddk] warn [NFC ERROR]NfcEstablishAuthCnxToServer: Failed to create new AuthD connection: Failed to connect to server 172.16.0.13:902
[22.10.2024 11:17:51.481] <  1436> vdl      | WARN|[vddk] warn [NFC ERROR]Nfc_BindAndEstablishAuthdCnx3: Failed to create a connection with server 172.16.0.13: Failed to connect to server 172.16.0.13:902
[22.10.2024 11:17:51.481] <  1436> vdl      | [vddk] NBD_ClientOpen: Couldn't connect to 10.250.100.13:902 Failed to connect to server 172.16.0.13:902

It's worth noting 172.16.0.0/24 is not accessible outside of VMWare (i.e. also unreachable for VEEAM).

The 10.250.100.0/24 is management network, the 172.16.0.0/24 is vMotion network. 

How can I make Veeam to connect to 10.250.100.0/24 and not 172.16.0.0/24?

During the vCenter "object" configuration in VEEAM, the connection from VEEAM to vCenter works perfectly (correct credentials, network visibility, ...)

Thanks in advance

Best answer by JMeixner

What is about standard gateways?

Sorry, but either your vCenter contacts the hosts on the wrong network or they are responding on the vMotion network. You say, you have no DNS, so I would check the gateway settings.

View original
Did this topic help you find an answer to your question?

13 comments

JMeixner
Forum|alt.badge.img+17
  • On the path to Greatness
  • 2650 comments
  • October 25, 2024

Which network is 172.16.0.x? Is this the vMotion network?
Or is it another network at the Veeam server?


  • Author
  • Not a newbie anymore
  • 7 comments
  • October 25, 2024

@JMeixner good point, added it to original answer as well. 10.250.100.0/24 is management and 172.16.0.0/24 is vMotion. 


JMeixner
Forum|alt.badge.img+17
  • On the path to Greatness
  • 2650 comments
  • October 25, 2024

The cause of most NFC errors fall into 4 primary categories:

  • DNS
       (The proxy or Veeam server cannot resolve the ESXi host)
  • Port (902)
       (The proxy can resolve the IP, but port 902 is blocked)
  • Permissions*
       (The account specified in [Backup Infrastructure] for the vCenter does not have permissions)
  • File Locks
       (The file Veeam is trying to read is locked within the vSphere environment)

 

Did you check the permissions of the user you are connecting to the vCenter?
https://helpcenter.veeam.com/docs/backup/permissions/cumulativepermissions.html?ver=120


coolsport00
Forum|alt.badge.img+20
  • Veeam Legend
  • 4143 comments
  • October 25, 2024

Hi @SZI -

You should be able to configure Preferred Networks in the Veeam Console:
https://helpcenter.veeam.com/docs/backup/vsphere/select_backup_network.html?ver=120


  • Author
  • Not a newbie anymore
  • 7 comments
  • October 25, 2024

Hi @coolsport00 , thanks for your feedback, however this doesn’t apply to my problem as my Veeam server has only one NIC and therefore it is defined which network to use to contact vCenter. Thank you nevertheless!


JMeixner
Forum|alt.badge.img+17
  • On the path to Greatness
  • 2650 comments
  • October 25, 2024

I have looked through my old notes.

A similar error was caused by wrong DNS configuration.

Please make sure, that the Veeam Server can resolve all ESX Server and the vCenter to the correct IP.


Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8495 comments
  • October 25, 2024

Based on the errors you posted the Veeam server appears to want to route through the vMotion network which is not correct so if the Veeam servers has a proper IP on the 10.250.100.0 network then as noted check DNS.  If you can ping the hosts and vCenter from the Veeam server and they resolve on the proper network things should work and if not then you might need to fix DNS or add host file entries to the Veeam server to have it take the proper routing to the VMware infrastructure.

 
 
 

  • Author
  • Not a newbie anymore
  • 7 comments
  • October 25, 2024

@JMeixner Thanks. To elaborate further on your previous post: 

  • VMware’s user used for VEEAM to connect to VMware is VMWare’s administrator, that should be sufficient 
  • the TCP port 902 is reachable on every ESXi host but not for vCenter (I believe this is correct)

When configuring the VMware infrastructure in Veeam, The only thing I enter to Veeam is DNS hostname for vCenter - this can be resolved by the VEEAM server (pinging to DNS name works well - vCenter is translated to correct IP address and responds to ping requests), I don’t define any of the ESXi hosts anywhere and ESXi hosts don’t have DNS name - only IP address.

 


Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8495 comments
  • October 25, 2024
SZI wrote:

@JMeixner Thanks. To elaborate further on your previous post: 

  • VMware’s user used for VEEAM to connect to VMware is VMWare’s administrator, that should be sufficient 
  • the TCP port 902 is reachable on every ESXi host but not for vCenter (I believe this is correct)

When configuring the VMware infrastructure in Veeam, The only thing I enter to Veeam is DNS hostname for vCenter - this can be resolved by the VEEAM server (pinging to DNS name works well - vCenter is translated to correct IP address and responds to ping requests), I don’t define any of the ESXi hosts anywhere and ESXi hosts don’t have DNS name - only IP address.

 

Then log in to your vCenter and check how it responds to ping for the hosts.  This could also be an issue going from VC to host.  I have seen this before if you don’t use DNS servers like in a homelab (I do have DNS set up).


JMeixner
Forum|alt.badge.img+17
  • On the path to Greatness
  • 2650 comments
  • October 25, 2024

In my understanding all ESXi Hosts and the vCenter server have to have an IP in the ESX Management LAN.

You are correct, there is a connection to the vCenter configured only. But this means that there is a problem between vCenter and ESX Hosts.


  • Author
  • Not a newbie anymore
  • 7 comments
  • October 25, 2024

@JMeixner The set-up is as you wrote, i.e. vCenter being in the same VLAN as are the MGMT interfaces of ESXi hosts.

If this helps, the set-up on the VMware (in terms of MGMT and vMotion networks coexistence) is as follows: 

(the screenshot is from one of three ESXi hosts, other hosts have it the same (with unique IPs per host, of course))

The response on ping from vCenter to hosts is with 0% packet drop (in tens of subsequent pings) and sub-millisecond response times. 

EDIT: 
While running a backup job on the VEEAM server (for the sake of simplicity only with one VM in it), I see this in wireshark (trace collected on the VEEAM server): 
 

I have struggles to understand why VEEAM is interested in the 172.16.0.0/24 network. The vCenter has hosts defined as follows: 

 


JMeixner
Forum|alt.badge.img+17
  • On the path to Greatness
  • 2650 comments
  • Answer
  • October 25, 2024

What is about standard gateways?

Sorry, but either your vCenter contacts the hosts on the wrong network or they are responding on the vMotion network. You say, you have no DNS, so I would check the gateway settings.


  • Author
  • Not a newbie anymore
  • 7 comments
  • October 25, 2024

@JMeixner No need to be sorry, on the contrary - your proposal to check gateways led me to answer. The vSphere backup NFC is enabled on the vMotion and disabled on the MGMT network. I need to check with the gentleman who set this cluster up what is the idea behind. Thanks. 


Comment