Skip to main content
  • EN

I’m a user of Veeam B&R Community Build 11.0.1.1261, and I’ve just started experiencing a problem a second time that I thought was fixed.

Some time ago (a couple of weeks to a month), I started experiencing an issue with Veeam backups. When it would try backing up, it would quickly error out with the following error:

Warning    Failed to synchronize rBackup Repository] Details: Failed to call RPC function 'NfsIsExist': Failed to reset previous connection exception: Permission denied.

 

The first time that happened, I assumed it was because I was disabling Veeam services that I thought were unnecessary. Despite re-enabling them, the error persisted. I eventually realized my installation of B&R was out of date, so I installed an updated version and that seemed to fix the issue for the time being.

Fast forward to today, and I’m seeing the aforementioned error again when trying to run backups. The repository is a Synology DiskStation, and I can verify that the share is accessible from a Linux host. I’m searching through the Veeam logs for something more helpful than the error above, but all I can find is a Task.Repository.log file which is actually less helpful (to me):

o02.12.2022 13:35:25] <14> Info         nCProxyRpcInvoker] RpcInvoker I29525276] has been created. Host: bhost:6160]
t02.12.2022 13:35:25] <14> Info         1NFS] Setting share: Share nfs3://SRepository host]:/volume1/Backup
02.12.2022 13:35:25] <14> Info         2NFS] Checking supported NFS protocol versions for sRepository host]
v02.12.2022 13:35:25] <14> Info         2CProxyRpcInvoker] RpcInvoker  29525276] was disposed
e02.12.2022 13:35:25] <14> Info         2CRepositoryScanProcessor] Checking container scan actuality. ContainerId: 0621ecc2-da41-42a4-9b51-6add65cf0ab3, repositoryId: 442f5777-7196-4750-b152-1a0dfa873b44
s02.12.2022 13:35:25] <14> Info         Rescanning container 0621ecc2-da41-42a4-9b51-6add65cf0ab3
c02.12.2022 13:35:25] <14> Info         6CProxyRpcInvoker] RpcInvoker 552590851] has been created. Host: Âhost:6160]
k02.12.2022 13:35:25] <14> Info         oNFS] Setting share: Share nfs3://1Repository host]:/volume1/Backup
Â02.12.2022 13:35:25] <14> Info         rNFS] Checking if NFS mount accessible on link nfs3://tRepository host]:/volume1/Backup
c02.12.2022 13:35:25] <14> Info         RCProxyRpcInvoker] RpcInvoker u52590851] was disposed
02.12.2022 13:35:25] <14> Error        Failed to investigate free space and total size for repository '2Repository]'
r02.12.2022 13:35:25] <14> Error        Failed to access NFS datastore (System.Exception)
202.12.2022 13:35:25] <14> Error           at Veeam.Backup.ProxyProvider.CProxyNfsFileCommander.GetStorageInfo(CNfsFcPath nfsPath)
o02.12.2022 13:35:25] <14> Error           at Veeam.Backup.Core.CNfsFileCommander.FindDirInfo(String link)
202.12.2022 13:35:25] <14> Error           at Veeam.Backup.Core.Repository.CDefaultRepositoryInvestigator.Investigate()
202.12.2022 13:35:25] <14> Error        Unable to investigate free space:
.02.12.2022 13:35:25] <14> Error        System.Exception: Failed to access NFS datastore
 02.12.2022 13:35:25] <14> Error           at Veeam.Backup.Core.Repository.CDefaultRepositoryInvestigator.Investigate()
N02.12.2022 13:35:25] <14> Error           at Veeam.Backup.ResourceScanner.CRepositoryContainerResourceProvider.RescanContainer(CRepositoryContainerResource containerResource)
402.12.2022 13:35:25] <14> Error           at Veeam.Backup.ResourceScanner.CRepositoryContainerResourceProvider.GetActualContainerResource(Boolean forcedScan)
[02.12.2022 13:35:25] <14> Error           at Veeam.Backup.ResourceScanner.CRepositoryScanProcessor.GetActualContainerResource(Boolean forcedScan)
l02.12.2022 13:35:25] <14> Error           at Veeam.Backup.ResourceScanner.CRepositoryScanProcessor.Execute(Boolean forcedScan)
e02.12.2022 13:35:25] <14> Info         Setting storage availability bit RepositoryUnavailable for repository 442f5777-7196-4750-b152-1a0dfa873b44
r02.12.2022 13:35:25] <14> Info         1ResourceScanner] Update resource  Repository] result: av. Unavailable, options
p02.12.2022 13:35:25] <14> Info         Prev result was Unavailable, options
02.12.2022 13:35:25] <14> Info         sResourceScanner] Scan is finished for resource lRepository]
e02.12.2022 13:35:25] <14> Info         tResourceScanner] Resourse scan period is 00:15:00 for repository [Repository].

 

I’m at a loss for where to go forward from here. Thanks in advance!

@Mildur 

I suppose you have already rebooted the Veeam Backup  and it's components?

Also if possible restart the Synology. I once had a case of hanging sessions on such a NAS.

I’d also add are you targeting it via DNS or IP address? Is the NFS server in the same LAN or at a remote site to the gateway server?

I suppose you have already rebooted the Veeam Backup  and it's components?

Also if possible restart the Synology. I once had a case of hanging sessions on such a NAS.

I have rebooted the host running Veeam B&R, correct. I did not think to reboot the NAS however!

I’d also add are you targeting it via DNS or IP address? Is the NFS server in the same LAN or at a remote site to the gateway server?

I am actually targeting the repository by DNS, and also authenticating the B&R host in NFS by DNS as well. Both hosts are on the same LAN, including DNS server.

 

I got an email this morning that a backup completed successfully. Sure enough, the repository is marked as accessible in Veeam too. There’s absolutely nothing I’ve changed since I gave up troubleshooting yesterday. Unfortunately the problem remains a mystery, although I like where @regnor is going in thinking that the DiskStation may have needed a reboot. Probably couldn’t hurt to update it to the latest DSM version, too.

Going forward it would be helpful if Veeam could be more clear about NFS errors, but for now all is well. Thank you all for the replies, I really appreciate it.

I’m a user of Veeam B&R Community Build 11.0.1.1261, and I’ve just started experiencing a problem a second time that I thought was fixed.

Some time ago (a couple of weeks to a month), I started experiencing an issue with Veeam backups. When it would try backing up, it would quickly error out with the following error:

Warning    Failed to synchronize rBackup Repository] Details: Failed to call RPC function 'NfsIsExist': Failed to reset previous connection exception: Permission denied.

 

The first time that happened, I assumed it was because I was disabling Veeam services that I thought were unnecessary. Despite re-enabling them, the error persisted. I eventually realized my installation of B&R was out of date, so I installed an updated version and that seemed to fix the issue for the time being.

Fast forward to today, and I’m seeing the aforementioned error again when trying to run backups. The repository is a Synology DiskStation, and I can verify that the share is accessible from a Linux host. I’m searching through the Veeam logs for something more helpful than the error above, but all I can find is a Task.Repository.log file which is actually less helpful (to me):

c02.12.2022 13:35:25] <14> Info         nCProxyRpcInvoker] RpcInvoker I29525276] has been created. Host: bhost:6160]
:02.12.2022 13:35:25] <14> Info         1NFS] Setting share: Share nfs3://SRepository host]:/volume1/Backup
h02.12.2022 13:35:25] <14> Info         2NFS] Checking supported NFS protocol versions for sRepository host]
e02.12.2022 13:35:25] <14> Info         2CProxyRpcInvoker] RpcInvoker  29525276] was disposed
r02.12.2022 13:35:25] <14> Info         2CRepositoryScanProcessor] Checking container scan actuality. ContainerId: 0621ecc2-da41-42a4-9b51-6add65cf0ab3, repositoryId: 442f5777-7196-4750-b152-1a0dfa873b44
i02.12.2022 13:35:25] <14> Info         Rescanning container 0621ecc2-da41-42a4-9b51-6add65cf0ab3
a02.12.2022 13:35:25] <14> Info         5CProxyRpcInvoker] RpcInvoker 552590851] has been created. Host: Âhost:6160]
e02.12.2022 13:35:25] <14> Info         sNFS] Setting share: Share nfs3://1Repository host]:/volume1/Backup
 02.12.2022 13:35:25] <14> Info         yNFS] Checking if NFS mount accessible on link nfs3://tRepository host]:/volume1/Backup
k02.12.2022 13:35:25] <14> Info         eCProxyRpcInvoker] RpcInvoker p52590851] was disposed
&02.12.2022 13:35:25] <14> Error        Failed to investigate free space and total size for repository '2Repository]'
Â02.12.2022 13:35:25] <14> Error        Failed to access NFS datastore (System.Exception)
.02.12.2022 13:35:25] <14> Error           at Veeam.Backup.ProxyProvider.CProxyNfsFileCommander.GetStorageInfo(CNfsFcPath nfsPath)
r02.12.2022 13:35:25] <14> Error           at Veeam.Backup.Core.CNfsFileCommander.FindDirInfo(String link)
.02.12.2022 13:35:25] <14> Error           at Veeam.Backup.Core.Repository.CDefaultRepositoryInvestigator.Investigate()
.02.12.2022 13:35:25] <14> Error        Unable to investigate free space:
D02.12.2022 13:35:25] <14> Error        System.Exception: Failed to access NFS datastore
 02.12.2022 13:35:25] <14> Error           at Veeam.Backup.Core.Repository.CDefaultRepositoryInvestigator.Investigate()
S02.12.2022 13:35:25] <14> Error           at Veeam.Backup.ResourceScanner.CRepositoryContainerResourceProvider.RescanContainer(CRepositoryContainerResource containerResource)
&02.12.2022 13:35:25] <14> Error           at Veeam.Backup.ResourceScanner.CRepositoryContainerResourceProvider.GetActualContainerResource(Boolean forcedScan)
002.12.2022 13:35:25] <14> Error           at Veeam.Backup.ResourceScanner.CRepositoryScanProcessor.GetActualContainerResource(Boolean forcedScan)
a02.12.2022 13:35:25] <14> Error           at Veeam.Backup.ResourceScanner.CRepositoryScanProcessor.Execute(Boolean forcedScan)
A02.12.2022 13:35:25] <14> Info         Setting storage availability bit RepositoryUnavailable for repository 442f5777-7196-4750-b152-1a0dfa873b44
c02.12.2022 13:35:25] <14> Info         2ResourceScanner] Update resource ÂRepository] result: av. Unavailable, options
s02.12.2022 13:35:25] <14> Info         Prev result was Unavailable, options
302.12.2022 13:35:25] <14> Info         uResourceScanner] Scan is finished for resource :Repository]
o02.12.2022 13:35:25] <14> Info         ResourceScanner] Resourse scan period is 00:15:00 for repository 0Repository].

 

I’m at a loss for where to go forward from here. Thanks in advance!

I had been having NFS issues with one of my clients as well. Just randomly started happening. Didn’t really reach a solution with support. They seemed to want to point the finger at VMware...I’m not sure if it’s a Veeam issue or not, at this point. It comes and goes and it just started for no reason. There’s not really any consistency to it, but it was happening fairly regularly.

@bp4JC if you’ve got some troubleshooting time, I’d take a packet capture of what’s happening. Sometimes it’s a lookup failure for DNS, but often it’s the NFS Server services not responding in a timely manner with stuff like Synology & QNAP. Unfortunately I find their solutions work like 98% of the time, creating lovely headaches like this!

@bp4JC if you’ve got some troubleshooting time, I’d take a packet capture of what’s happening. Sometimes it’s a lookup failure for DNS, but often it’s the NFS Server services not responding in a timely manner with stuff like Synology & QNAP. Unfortunately I find their solutions work like 98% of the time, creating lovely headaches like this!

That’s a great idea but it’s just so random that I am not sure how I can determine when to run the packet capture. It was happening for about two or three weeks, and then it suddenly stopped. To be fair, I know that my CTO was doing some work on the VMware environment around the time it stopped and I don’t think I saw the issue pop up at all last week, but it could very well start again. I’ll have to look into trying that if it starts again. Thanks for chiming in on this!

@bp4JC if you’ve got some troubleshooting time, I’d take a packet capture of what’s happening. Sometimes it’s a lookup failure for DNS, but often it’s the NFS Server services not responding in a timely manner with stuff like Synology & QNAP. Unfortunately I find their solutions work like 98% of the time, creating lovely headaches like this!

That’s a great idea but it’s just so random that I am not sure how I can determine when to run the packet capture. It was happening for about two or three weeks, and then it suddenly stopped. To be fair, I know that my CTO was doing some work on the VMware environment around the time it stopped and I don’t think I saw the issue pop up at all last week, but it could very well start again. I’ll have to look into trying that if it starts again. Thanks for chiming in on this!

Drop me a DM if I can help when it’s flaring up, if I’m about I’ll try!

@bp4JC if you’ve got some troubleshooting time, I’d take a packet capture of what’s happening. Sometimes it’s a lookup failure for DNS, but often it’s the NFS Server services not responding in a timely manner with stuff like Synology & QNAP. Unfortunately I find their solutions work like 98% of the time, creating lovely headaches like this!

That’s a great idea but it’s just so random that I am not sure how I can determine when to run the packet capture. It was happening for about two or three weeks, and then it suddenly stopped. To be fair, I know that my CTO was doing some work on the VMware environment around the time it stopped and I don’t think I saw the issue pop up at all last week, but it could very well start again. I’ll have to look into trying that if it starts again. Thanks for chiming in on this!

If you don’t mind me asking, is this VMWare providing the NFS service? Trying to find some thread tying my DiskStation to your situation.

Just want to update everyone. I started experiencing this issue again, right around when a power outage occurred and my DiskStation went through it’s UPS shutdown procedure. I’ve executed a full reboot on the DiskStation, and after coming back up, backups are working fine now. This is with the DiskStation Manager software version DSM 6.2.4-25556 Update 6 (latest available as of now.) I haven’t upgraded to 7 yet so I probably need to do that.

Comment


Terms & Conditions