Error: Failed to call RPC function 'FcReadFileEx': An unexpected network error occurred. Failed to read data from file

Based on the error message it would seem the VBM file is corrupted.  Did you try to run an Active Full backup?

What version of Veeam are you using?  What version of Windows?  Did you check the backup logs for the job here - C:\ProgramData\Veeam\Backup\Jobname?

You can also try to remove this VBM file, rescan the repository and then try to run the policy gain and see if it works.

Others have addressed the VBM appears to be corrupted, I’d like to share some details to attempt to prevent this from happening again.

• You’ll have a gateway server talking to your SMB share, make sure this gateway server is in the same physical location as the NAS and can talk directly to it without any firewalls or traffic inspection in between.
• Make sure Synology’s write cache feature is disabled. Performance will unfortunately take a hit but it means that writes won’t be confirmed until they’ve actually been written to disk instead of to RAM (as if you have a crash or power loss the data is inconsistent!, Synology doesn’t have any battery backups for the write cache).
• Id suggest using iSCSI to communicate to the Synology instead of SMB if possible. This way you can leverage fast clone to increase storage efficiency. Your server that is connected to the storage would have the storage mapped as a drive letter with an ReFS file system (assuming you’re using windows here, XFS if Linux), you’d also be able to leverage multi path IO to increase network speed between source and destination potentially if they’ve both got multiple network ports. Finally you can also put that traffic on a dedicated VLAN or switch to avoid noisy network issues.

I keep forgetting about doing this with the VBM file.  This is another great catch by @wesmrt 

Several things may be at play here @MarkD . And each could be causing issue with your metadata file (.vbm), as Chris states.

If your backend Repository storage has “unraid” settings, you could try and change the setting for “Case-sensitive names” from Force lower to Auto, then change the .vbm file name (keep the name) to have all upper case letters. See a comment from this Comm Hub post.

Another issue could be your antivirus software scanning your storage (Repo) volume during backup, causing “network” and/or access issue to your .vbm file, as noted in this Comm Hub post. You can either disable A/V on your Repo and/or VBR server, or create exceptions for your Repo paths.

Another potential cause could be access credentials. Verify you’re using domain\username as the format to connect to your backend storage. As others noted, you can do this in the Properties of your Repo and do a rescan operation.

Hopefully one of those options work for you.

Hello all!

all the tricks and tips did not work out for me.

its so frustrating.. i even did add a new networkcard in de VM with the E1000 Nic instead of the VMX standard.. and even that will not work… also checked the network on switches etc if there is someting wrong. the Jumbo Frames are all OK on the whole route.

it seems that network segment can't reach the Synology NAS. while it allready works for years.

suddenly it stops working a 1.5 week ago..

even i did try the veeam.backup.extracter.exe but.. my backup files are over 1TB and i simple has no room  to copy all the data into 1 and the same folder.

im also had restarted the vCenter appliance.. because it communicate with that too but no result.

also the backup VM restarted and also no result.

someone still a tip or maby i did overlook someting?

here i have a copy paste part where the error started:

Info         VC session cookie is '"f79023ab0a2032c4e818dcef7b59a8126e463695"', url 'https://dca-vm-133:443/sdk'.
Info         [Soap] Host content info: host "dca-vm-133", type "vpx", version "6.0.0", build "4541948", apiVersion "6.0", hostTime "11-2-2024 18:58:47", sessionKey: "52ea2a83-1855-cd5b-bf16-bc8e76bfac26"
Info         [Soap] Connection 'dca-vm-133:443:oss\svc_ve:False::0:1' is disposing.
Info         Job size: 13,3 TB (7,1 TB provisioned).
Info         [JobSession] TotalObjects '68', totalSize '13,3 TB'
Info         [BackupJobTarget] Initializing backup target, mode: 'Retry', agent sharing mode: 'SharedAgent', repository: 'Backup Repository'.
Info         [ProxyAgentTracker] Stopping agents on Job: '01735064-3696-45db-b43b-2b59d52dfc6f'
Info         Checking version of repository 'Backup Repository LCM'.
Info         Repository 'Backup Repository LCM' version is supported.
Info         [RPC] Loading options.
Info         Resolved by NTLM strategy ip addresses and host names: xxx.xxx.xxx.xxx xxx.xxx.xxx.xxx bac-vm-112
Info         [CProxyRpcInvoker] RpcInvoker [39838562] has been created. Host: [bac-vm-112:6160]
Info         [RPC] RPC confirm session has been created. Id: {764c416d-a891-4c58-aa48-c882e3fa920e}
Info         [BackupEncryptedStateUpdater] Initializing backup encryption state (BackupId='3c0c8db3-af0f-460f-8d06-1a67c407d1a5', IsEncryptionEnabled=False, HasCryptedStorages=False, EncryptionState=Unencrypted)
Info         [BackupLock] Locking 1 backup(s) as write-only: 3c0c8db3-af0f-460f-8d06-1a67c407d1a5
Info             [LockResolver] Getting conflicting sessions
Info             [LockResolver] No conflicting sessions found
Info             [API] Creating new lease. TTL: [00:05:00]
Info             [API] New lease was created. Id: [50251510-3496-4229-b305-301683d242c6], TTL: [00:05:00]
Info         [LeaseKeeper] Created. LeaseId: [5025151034964229b305301683d242c6], TTL: [300 sec], ignoreExceptions: [False].
Info         [BackupLock] Backup(s) have been locked, lockIds: [9e18c1c3-6456-45f0-a5d5-f776be2a5fd5]
Info         [CStorageLinksHelper] Asserting storage links...
Info         [CStorageLinksHelper] Storage links checked successfully
Info         [Mutex] Created mutex [Global\BackupMeta[01735064-3696-45db-b43b-2b59d52dfc6f]]
Info         [Mutex] Acquired mutex [Global\BackupMeta[01735064-3696-45db-b43b-2b59d52dfc6f]]
Info         Removing dca-vm-133:443:oss\svc_ve:False::0:1 from cache. Reason: Expired
Info         [Soap] Logout from "https://dca-vm-133:443/sdk"
Info         [FC] Loading content from file '\\xxx.xxx.xxx.xxx\NMA B2D ALL.vbm'
Error        Failed to call RPC function 'FcReadFileEx': An unexpected network error occurred. Failed to read data from file [\\NMA B2D ALL.vbm].
Error        An unexpected network error occurred.
Error        Failed to read data from file [\\NMA B2D ALL.vbm].
Error        --tr:Error code: 0x0000003b
Error        --tr:Failed to complete RPC binary request. Method: [FcReadFileEx], In XML: [<InputArguments><FilePath value="\\NMA B2D ALL.vbm" /><DesiredAccess value="2147483648" /><ShareMode value="1" /><CreationDisposition value="3" /><FlagsAndAttrs value="0" /><Offset value="0" /><BytesToRead value="65536" /></InputArguments>]
Error        --tr:Failed to invoke RPC request
Error        --tr:Failed to call DoRpc. CmdName: [DoSerialRpc] inParam: [<InputArguments/>].
Error        --tr:Failed to call DoRpcWithBinary. Method: [DoSerialRpc], In xml: [<InputArguments/>]
Error        --tr:Failed to call DoRpcWithBinary
Error        --tr:Failed to do RPC call with serialized data, Strategy type: DefaultStrategy
Error        --tr:Failed to call [DoSerialRpc], session id: {764c416d-a891-4c58-aa48-c882e3fa920e}
Error        --tr:Failed to call [FcReadFileEx] (System.Runtime.InteropServices.COMException)
Error           at VeeamProxyClient.GenerateComException(basic_string<char\,std::char_traits<char>\,std::allocator<char> >* excMsg, Int32 errHrCode)
Error           at VeeamProxyClient.RegenerateCLRException(exception* excBase)
Error           at VeeamProxyClient.CCliProxyRpcInvoker.DoRpcWithBinaryEx(String cmdName, String inXml, Byte[]& inData, String& outXml, Byte[]& outData)
Error           at Veeam.Backup.ProxyProvider.CProxyRpcInvoker.CallWithBinaryEx(String methodName, CProxyInvokeInputArg inputArgs, Byte[]& inData, Byte[]& outData)
Error        Failed to check whether meta file [\\xxx.xxx.xxx.xxx\NMA B2D ALL.vbm] is synchronized with DB
Error        Failed to load content from file '\\xxx.xxx.xxx.xxx\NMA B2D ALL.vbm' (System.Exception)
Error           at Veeam.Backup.ProxyProvider.CProxyFileCommander.LoadFileContentBinary(String link, UInt64 maxSize)
Error           at Veeam.Backup.Core.CWinFileCommander.LoadFileContentBinary(String link, UInt64 maxSize)
Error           at Veeam.Backup.Meta.CBackupMetaFile.LoadContent(UInt64 maxBytesToRead)
Error           at Veeam.Backup.Meta.CBackupMetaHeaderReader..ctor(IBackupMetaFileEx meta)
Error           at Veeam.Backup.Core.CBackupMetaSynchronizationChecker.IsMetaFilesSynchronizedWithDb(CBackup backup, CLazyBackupMetaFile[] metasOnFs)
Error        An unexpected network error occurred. (Veeam.Backup.Common.CCppCOMException)
Error        Failed to read data from file [\\xxx.xxx.xxx.xxx\NMA B2D ALL.vbm]. (Veeam.Backup.Common.CCppComponentException)
Error           in c++: Error code: 0x0000003b

Hi @MarkD -

The only thing I can see from your log output is Veeam seems to be having a hard time, or is unable to, read your metadata file (the .vbm file, named: NMA B2D ALL.vbm ), as others have shared above. Is this file even present in your Repository storage? If so, then it just may be corrupted and as others have shared, you can remove the file, then perform a rescan of your Repository to recreate it; then reattempt your job. If you’re still having problems, I think we all may have exhausted all our knowledge and just recommend you get in touch with Veeam Support...unless someone has other thoughts?

Sometimes ‘little’ things happen MarkD. Glad you got it all sorted 😊

Don’t sweat the little things as they say.  Glad you figured it out and resolved the issue.