Hi everyone, just want to post this if anyone seen something like this.
B&R 12.2.0.334 Enterprise Plus subscription running on Windows Server 2022 Datacenter.
Target VMs run on Windows Server 2022 Datacenter Hyper-V.
Backing up VMs that are on local subnet works, except couple servers that are on different subnet.
c:\programdata\veeam\backup log doesn’t seem to show anything particular of interest.
It’s backing up to local NAS WD PR4100.
Email notification error shows An existing connection was forcibly closed by the remote host, which is not mentioned in the log file.
Windows firewall enabled exception all tcp/udp from both subnet. I can do these things to target servers:
- ping
- both access \\<fqdn>\c$ and \\<ip_address>\c$
- add them to the infrastructure
- install transport/integration services from B&R
I rebooted them all, reduce concurrent tasks to 2.
I just want to make sure it’s not part of my configuration, if not, then I’ll escalate it to our org’s network people.
Hi @perkedel -
Welcome to the Community. Where is the error occuring?...on all VMs, local subnet VMs, or remote subnet VMs? Have you ever been able to back them up? Or, have you been able to back them up and just now receiving error? Is there a reason you haven’t upgraded to 12.3 yet?
I’ve seen this error for several reasons → Windows Firewall, Antivirus causing issues (make sure you have exclusions config’d according to: https://www.veeam.com/kb1999 ), acct pwd has changed or expired, network interruption (maintenance and/or update maybe?), stopped Veeam service just to name a few.
Please share more details on what you’re doing as well as provide info to my questions and we’ll try to assist.
Best.
Hi @perkedel, kindly take a look at the service to see if it is running!
Thanks guys. All the services are running. I’m upgrading B&R to 12.3.0.310 and will retry again.
The errors are occurring when I run backup jobs on certain VMs that is on remote subnet. VMs that are on local subnet as B&R works fine.
I was able to backed them up before, but the servers were on WS 2016. The same servers are now on WinServer 2022 just as the B&R.
For firewall exclusion, I added the entire subnet for any incoming/outgoing. Probably not network interruption as it happens every time I run the backup on manual regardless of time. I can still access the remote VMs from the B&R, just not the backup jobs.
Hi and welcome @perkedel,
The issue can also be related to live migrations done by a cluster or by node outages (maybe updates?) so that only for a certain amount of time (your backup window) one of the cluster nodes is not available.
You have to keep in mind that with Hyper-V you usually use on-host proxies so that the availability of the Hyper-V nodes is mandatory.
Best
Lukas
Thank you everyone for replying. So here’s what I tried:
Upgraded to 12.3.0.310.
Uninstall the previous version, reboot, reinstall, reboot.
Restore config from backup, retry all the jobs all work except that one.
I also tried:
- Added iSCSI back up repo to the target.
- Changing to On host, Off-host to local subnet, Off-host to remote subnet.
Remote firewall all open for these:
- Veeam Backup Hyper-V Integration Service (In)
- Veeam Data Mover (Veeam Transport Service) (In)
- Veeam Data Mover x64 (Veeam Transport Service) (In)
- Veeam Guest Interaction Proxy (In)
- Veeam Installer Service (In)
- Veeam Transport Service (In)
Okay this is a weird problem, I have no idea. Quick glance at log:
oTlsSocket] Connecting to 'server_public_ip_here'.
oTlsSocket] Error on authentication. Thumbprint on begin: the_thumbprint_here, Thumbprint on end: the_thumbprint_here,
Error message: An existing connection was forcibly closed by the remote host
Failed to connect to agent ‘server_fqdn_here, EP ‘server_public_ip_here:2500' (System.Exception)
An existing connection was forcibly closed by the remote host (System.Net.Sockets.SocketException)
Maybe authentication issue from what I see on TlsSocket]. I use the same user/pass saved in the datacenter password in B&R settings.
Would TLS version could cause an issue? Maybe a certificate name mismatch error. 
@perkedel You mentioned “server_public_ip_here” in the logs. Are you using local IP or public IP for the Hyper-V hosts or Veeam components to connect? It should be using local IP’s of the Hyper-V host and proxy servers.
Try to run the connection test from the VBR server to the Hyper-V host and see the results
Test-NetConnection -ComputerName <HyperV_ip> -Port 2500
Also make sure TLS 1.2 is enabled in hyper-v hosts
Thanks for the tip on that test connection @Mohamed Ali
We do not have local IP (192.x), all hosts/vms use public IP addresses.
I tried Test-NetConnection and the test failed. Also I tried it on the hosts that the backups do work, the test came out failed. So the test failed on both hosts that the backup works and hosts that the backup does not work, either using IP address and FQDN.
TLS1.2 should be on by default on Server 2022 according to MSFT, and I visually confirmed the Registry settings are on the host/vm on the one that doesn’t work.
I also checked the cert is good.
thank you everyone who replied.
there’s no longer tlssocket error like before but the job still fails. I have no idea what else to try so I’ve submitted a case.
Feel free to keep us posted when Support is reacting!
So it seems that we found there’s a WAN side that is blocking port 2500. It’s been unblocked on the WAN side firewall and B&R is working now, thank you everyone.
Glad to hear that you were able to figure this out.
Take care!
Lukas
Glad to hear everything is going well! However, keep in mind that Veeam uses a range of ports to function properly. If you encounter similar issues, you can check this link for more details:
https://helpcenter.veeam.com/docs/backup/vsphere/used_ports.html?ver=120
