Question

Backup Failed physical server


Userlevel 2
  • Not a newbie anymore
  • 9 comments

Error: Incorrect function. Asynchronous read operation failed Failed to upload disk. Agent failed to process method {DataTransfer.SyncDisk}. Exception from server: Incorrect function. Unable to retrieve next block transmission command. Number of already processed blocks: [6170]. Failed to download disk '2bc23b75-36bc-4dc6-a7a6-0a0896103c88


26 comments

Userlevel 7
Badge +20

When posting in the community please try to provide more information other than just an error message from the job/console.

  1. What Hypervisor are you using?
  2. What storage are you using - Production and Backup Repository?
  3. What version of Windows are you using?
  4. What version of Veeam are you using?
  5. What task are you trying to do that is throwing the error?
  6. What did the logs say - C:\ProgramData\Veeam\Backup\JobName
  7. What other information can you tell us about the issue?

Please update this post with this information for us to further help.

Userlevel 2

I have share error message

Error: Incorrect function. Asynchronous read operation failed Failed to upload disk. Agent failed to process method {DataTransfer.SyncDisk}. Exception from server: Incorrect function. Unable to retrieve next block transmission command. Number of already processed blocks: [6170]. Failed to download disk '2bc23b75-36bc-4dc6-a7a6-0a0896103c88

Userlevel 2

Microsoft Windows Server 2012 R2 Standard Edition, 64-bit (build 9600)

Userlevel 7
Badge +20

The error message does not help us!  Please answer all the questions I posted as we need to know what actions your are doing to cause the error.  Thanks for the Windows version but that does not help either we need as many details as possible not just the error.

Userlevel 2
  1. What Hypervisor are you using? NA
  2. What storage are you using - Production and Backup Repository? Qnap NAS Share Drive SMB
  3. What version of Windows are you using?
  4. What version of Veeam are you using? 12 
  5. What task are you trying to do that is throwing the error? 
  6. What did the logs say - C:\ProgramData\Veeam\Backup\JobName
  7. What other information can you tell us about the issue?
Userlevel 7
Badge +20
  1. What Hypervisor are you using? NA
  2. What storage are you using - Production and Backup Repository? Qnap NAS Share Drive SMB
  3. What version of Windows are you using?
  4. What version of Veeam are you using? 12 
  5. What task are you trying to do that is throwing the error? 
  6. What did the logs say - C:\ProgramData\Veeam\Backup\JobName
  7. What other information can you tell us about the issue?

So, if you are not using a Hypervisor - is this an Agent backup job?  Still no answer to questions 5 & 7 which help.  What are you specifically trying to do?  Are you backing up a physical server?  You can see how it is hard to help without all the required information.

Userlevel 2

[08.01.2024 09:40:43.746] <01> Info         -------------------------------------------------------------------------------------
[08.01.2024 09:40:43.747] <01> Info         STARTEPAGENTPOLICYJOB
[08.01.2024 09:40:43.747] <01> Info         owner=[vbsvc]
[08.01.2024 09:40:43.747] <01> Info         Normal
[08.01.2024 09:40:43.747] <01> Info         e7ac40d1-5743-4bfe-a958-33c3c15ca4df
[08.01.2024 09:40:43.747] <01> Info         ed3c2cd6-496c-466b-afd0-c568fee937ff
[08.01.2024 09:40:43.747] <01> Info         -------------------------------------------------------------------------------------
[08.01.2024 09:40:43.759] <01> Info         [CJobLogsManager] Cannot find quota. JobId : e7ac40d1-5743-4bfe-a958-33c3c15ca4df.
[08.01.2024 09:40:43.840] <01> Info         Starting job mode: 'Normal'.
[08.01.2024 09:40:43.842] <01> Info         [Session] Id 'ed3c2cd6-496c-466b-afd0-c568fee937ff', State 'Working'.
[08.01.2024 09:40:43.994] <01> Info         Add process cloud feature DBOSExtentCache
[08.01.2024 09:40:44.026] <01> Info         Custom command execution (pre script) disabled
[08.01.2024 09:40:44.117] <01> Info         Cached target repository 88788f9e-d8f5-4eb4-bc4f-9b3f5403bcec
[08.01.2024 09:40:44.117] <01> Info         [CCloudConnectionHolder] Target repository is not CloudRepository
[08.01.2024 09:40:44.396] <01> Info         [EpAgentBackupJobPerformer] Policy task accepted. Policy ID 'e7ac40d1-5743-4bfe-a958-33c3c15ca4df', Type 'ProcessAll', Mode 'Normal'
[08.01.2024 09:40:44.403] <01> Info         Start acquiring move copy mutex for e7ac40d1-5743-4bfe-a958-33c3c15ca4df
[08.01.2024 09:40:44.406] <01> Info         Creating mutex Global\MoveCopyJobBackupJobSynchronization[e7ac40d1-5743-4bfe-a958-33c3c15ca4df]
[08.01.2024 09:40:44.407] <01> Info         Finish acquiring move copy mutex for e7ac40d1-5743-4bfe-a958-33c3c15ca4df
[08.01.2024 09:40:45.014] <01> Info         Existing memberships with:
[08.01.2024 09:40:45.015] <01> Info             Agent 63275e48-791f-4484-9b18-f7e41b25c868
[08.01.2024 09:40:45.175] <01> Info         -------------------------------------------------------------------------------------
[08.01.2024 09:40:45.175] <01> Info             Job ID: [e7ac40d1-5743-4bfe-a958-33c3c15ca4df]
[08.01.2024 09:40:45.175] <01> Info             Job Name: [TJDC01]
[08.01.2024 09:40:45.175] <01> Info             Job Mode: [Normal]
[08.01.2024 09:40:45.175] <01> Info             Job Type: [Files EpAgentBackup]
[08.01.2024 09:40:45.175] <01> Info             Job Target Type: [Other]
[08.01.2024 09:40:45.176] <01> Info             Job Options: [<JobOptionsRoot><RunManually>True</RunManually><GroupSnapshotProcessing>True</GroupSnapshotProcessing><RetentionType>1</RetentionType><VbrAuthenticationMode>0</VbrAuthenticationMode><PolicyType>1</PolicyType><PolicySourceType>2</PolicySourceType><PolicyMacSourceType>0</PolicyMacSourceType><PolicyUnixSourceType>0</PolicyUnixSourceType><SourceFsItemsScope>0</SourceFsItemsScope><BackupAllUsbDrives>False</BackupAllUsbDrives><BackupSystemState>False</BackupSystemState><ExcludeSystemState>False</ExcludeSystemState><BackupSpecifiedItems>True</BackupSpecifiedItems><ExcludeOneDriveFolders>False</ExcludeOneDriveFolders><TargetShareType>0</TargetShareType><VbrAddress>TJDC02.BAILHOTLINE.COM</VbrAddress><VbrPort>10005</VbrPort><IsSnapshotlessMode>False</IsSnapshotlessMode><RetainDaysToKeep>7</RetainDaysToKeep><PolicyDestType>0</PolicyDestType><BackupCacheEnabled>False</BackupCacheEnabled><BackupCachePathSelectionMode>0</BackupCachePathSelectionMode><BackupCachePath /><BackupCacheSize>10</BackupCacheSize><BackupCacheSizeUnit>3</BackupCacheSizeUnit><EnableFullBackup>False</EnableFullBackup><StgBlockSize>KbBlockSize1024</StgBlockSize><EnableDeduplication>True</EnableDeduplication><CompressionLevel>5</CompressionLevel><StorageEncryptionEnabled>False</StorageEncryptionEnabled><DirtyBlocksNullingEnabled>True</DirtyBlocksNullingEnabled><FullBackupScheduleKind>Daily</FullBackupScheduleKind><FullBackupMonthlyScheduleOptions><DayNumberInMonth>First</DayNumberInMonth><DayOfWeek>Monday</DayOfWeek><DayOfMonth>1</DayOfMonth><Months><EMonth>January</EMonth><EMonth>February</EMonth><EMonth>March</EMonth><EMonth>April</EMonth><EMonth>May</EMonth><EMonth>June</EMonth><EMonth>July</EMonth><EMonth>August</EMonth><EMonth>September</EMonth><EMonth>October</EMonth><EMonth>November</EMonth><EMonth>December</EMonth></Months></FullBackupMonthlyScheduleOptions><TransformFullToSyntethic>True</TransformFullToSyntethic><SnmpNotification>False</SnmpNotification><EmailNotification>False</EmailNotification><EmailNotificationAddresses></EmailNotificationAddresses><UseCustomEmailNotificationOptions>False</UseCustomEmailNotificationOptions><EmailNotifyOnceADay>False</EmailNotifyOnceADay><EmailNotifyTime>12/21/2023 22:00:00</EmailNotifyTime><EmailNotificationSubject>[%JobResult%] %JobName% (%ObjectCount% objects) %Issues%</EmailNotificationSubject><EmailNotifyOnSuccess>True</EmailNotifyOnSuccess><EmailNotifyOnWarning>True</EmailNotifyOnWarning><EmailNotifyOnError>True</EmailNotifyOnError><EmailNotifyOnLastRetryOnly>True</EmailNotifyOnLastRetryOnly><EmailNotifyOnWaitingTape>True</EmailNotifyOnWaitingTape><PostJobCommand><Periodicity>Cycles</Periodicity><PreScriptEnabled>False</PreScriptEnabled><PreScriptCommandLine></PreScriptCommandLine><Frequency>1</Frequency><Enabled>False</Enabled><CommandLine></CommandLine><Days><Day>Saturday</Day></Days></PostJobCommand><GenerationPolicy><EnableRecheck>False</EnableRecheck><EnableSimpleObjectStorageRecheck>True</EnableSimpleObjectStorageRecheck><RecheckScheduleKind>Monthly</RecheckScheduleKind><RecheckMonthlyScheduleOptions><DayNumberInMonth>Last</DayNumberInMonth><DayOfWeek>Saturday</DayOfWeek><DayOfMonth>1</DayOfMonth><Months><EMonth>January</EMonth><EMonth>February</EMonth><EMonth>March</EMonth><EMonth>April</EMonth><EMonth>May</EMonth><EMonth>June</EMonth><EMonth>July</EMonth><EMonth>August</EMonth><EMonth>September</EMonth><EMonth>October</EMonth><EMonth>November</EMonth><EMonth>December</EMonth></Months></RecheckMonthlyScheduleOptions><RecheckTime>05:00:00</RecheckTime><EnableCompactFull>False</EnableCompactFull><EnableCompactFullLastTime /><CompactFullBackupScheduleKind>Monthly</CompactFullBackupScheduleKind><CompactMonthlyScheduleOptions><DayNumberInMonth>Last</DayNumberInMonth><DayOfWeek>Saturday</DayOfWeek><DayOfMonth>1</DayOfMonth><Months><EMonth>January</EMonth><EMonth>February</EMonth><EMonth>March</EMonth><EMonth>April</EMonth><EMonth>May</EMonth><EMonth>June</EMonth><EMonth>July</EMonth><EMonth>August</EMonth><EMonth>September</EMonth><EMonth>October</EMonth><EMonth>November</EMonth><EMonth>December</EMonth></Months></CompactMonthlyScheduleOptions><DeletionGuardDays>21</DeletionGuardDays><UseDeletionGuardDays>True</UseDeletionGuardDays><RecheckDays><DayOfWeek>Saturday</DayOfWeek></RecheckDays><CompactFullBackupDays><DayOfWeek>Saturday</DayOfWeek></CompactFullBackupDays></GenerationPolicy><EnableDeletedVmDataRetention>False</EnableDeletedVmDataRetention><RetainDays>30</RetainDays><UseSanSnapshots>False</UseSanSnapshots><SanStorageProxyAutoDetect>True</SanStorageProxyAutoDetect><FailoverFromSan>True</FailoverFromSan><AgentProcessMissingWarningEnabled>False</AgentProcessMissingWarningEnabled><AgentProcessMissingValidPeriodDays>7</AgentProcessMissingValidPeriodDays><GfsPolicy IsEnabled="False"><Weekly IsEnabled="False" KeepBackupsForNumberOfWeeks="1" DesiredTime="0" /><Monthly IsEnabled="False" KeepBackupsForNumberOfMonths="1" DesiredTime="1" /><Yearly IsEnabled="False" KeepBackupsForNumberOfYears="1" DesiredTime="1" /></GfsPolicy><PublicCloudType>0</PublicCloudType><TransformToSyntethicMonthlyScheduleOptions /><IncludeFsItems><Item Type="0">C:\</Item></IncludeFsItems><ExcludeFsItems /><IncludeFileTags /><IncludeMasks /><ExcludeMasks /><FullBackupDays><DayOfWeek>Saturday</DayOfWeek></FullBackupDays><TransformToSyntethicDays><DayOfWeek>Saturday</DayOfWeek></TransformToSyntethicDays></JobOptionsRoot>]
[08.01.2024 09:40:45.212] <01> Info             Job VSS Settings: [VssSnapshotOptions: [Enabled: [True], ApplicationProcessingEnabled: [True], IgnoreErrors: [False], IsCopyOnly: [False], UsePersistentGuestAgent: [False]], WinGuestFSIndexingOptions: [Type: [None]], LinGuestFSIndexingOptions: [Type: [None]], SqlBackupOptions: [TransactionLogsProcessing: [TruncateOnlyOnSuccessJob], BackupLogsFrequencyMin: [15], UseDbBackupRetention: [True], RetainDays: [15], ProxyAutoSelect: [True]FailJobOnDbAbsenceOrBackupImpossibility: [False]], ExchangeBackupOptions: [TransactionLogsProcessing: [TruncateOnlyOnSuccessJob]], WinCredsId: [00000000-0000-0000-0000-000000000000], LinCredsId: [00000000-0000-0000-0000-000000000000], Old Properties: [Enabled: [True], IgnoreErrors: [False], GuestFSIndexingType: [None], IncludedIndexingFolders: [], ExcludedIndexingFolders: [], LinGuestFSIndexingType: [None], LinIncludedIndexingFolders: [], LinExcludedIndexingFolders: [], IsFirstUsage: [True]]]
[08.01.2024 09:40:45.228] <01> Info             Job ScheduleOptions: [Start time: [1/8/2024 10:00:00 PM], Latest run time: [1/8/2024 8:03:22 AM], Timeout for backup completion: [180min], Next run time: [], Retry times on failure: [3], Retry timeout: [10 min], Daily options: [Enabled: True, DayNumberInMonth: Everyday, Days: Sunday, Monday, Tuesday, Wednesday, Thursday, Friday, Saturday]]
[08.01.2024 09:40:45.267] <01> Info             Job AutoScheduleOptions: [<AutoScheduleOptions PerformActiveFullBackup="False" ActiveFullBackupKind="0" ActiveFullBackupDays="Saturday;" PerformTransformToSyntethic="True" TransformToSyntethicKind="0" TransformToSyntethicDays="Saturday;"><ActiveFullBackupMonthlyOption><DayNumberInMonth>First</DayNumberInMonth><DayOfWeek>Monday</DayOfWeek><DayOfMonth>1</DayOfMonth><Months><EMonth>January</EMonth><EMonth>February</EMonth><EMonth>March</EMonth><EMonth>April</EMonth><EMonth>May</EMonth><EMonth>June</EMonth><EMonth>July</EMonth><EMonth>August</EMonth><EMonth>September</EMonth><EMonth>October</EMonth><EMonth>November</EMonth><EMonth>December</EMonth></Months></ActiveFullBackupMonthlyOption><TransformToSyntethicMonthlyOption><DayNumberInMonth>First</DayNumberInMonth><DayOfWeek>Monday</DayOfWeek><DayOfMonth>1</DayOfMonth><Months><EMonth>January</EMonth><EMonth>February</EMonth><EMonth>March</EMonth><EMonth>April</EMonth><EMonth>May</EMonth><EMonth>June</EMonth><EMonth>July</EMonth><EMonth>August</EMonth><EMonth>September</EMonth><EMonth>October</EMonth><EMonth>November</EMonth><EMonth>December</EMonth></Months></TransformToSyntethicMonthlyOption></AutoScheduleOptions>]
[08.01.2024 09:40:45.275] <01> Info         Enable parallel processing checkbox is on
[08.01.2024 09:40:45.322] <01> Info             Object in job: [Id: [ff216d9a-0668-43b2-8f7f-d9b2c7646426], ObjectId: [524f5321-b06e-41cb-a865-cfbc13743aee], JobId: [e7ac40d1-5743-4bfe-a958-33c3c15ca4df]FolderId: [00000000-0000-0000-0000-000000000000]
[08.01.2024 09:40:45.322] <01> Info         Location: [192.168.35.2], Type: [Include], ApproxSize: [0]
[08.01.2024 09:40:45.322] <01> Info         VssOptions: [VssSnapshotOptions: [Enabled: [True], ApplicationProcessingEnabled: [True], IgnoreErrors: [False], IsCopyOnly: [False], UsePersistentGuestAgent: [False]], WinGuestFSIndexingOptions: [Type: [ExceptSpecifiedFolders], IncludedFolders: [], ExcludedFolders: [%windir%; %ProgramFiles%; %ProgramFiles(x86)%; %ProgramW6432%; %TEMP%]], LinGuestFSIndexingOptions: [Type: [ExceptSpecifiedFolders], IncludedFolders: [], ExcludedFolders: [/cdrom; /dev; /media; /mnt; /proc; /tmp; /lost+found]], SqlBackupOptions: [TransactionLogsProcessing: [TruncateOnlyOnSuccessJob], BackupLogsFrequencyMin: [15], UseDbBackupRetention: [True], RetainDays: [15], ProxyAutoSelect: [True]FailJobOnDbAbsenceOrBackupImpossibility: [False]], ExchangeBackupOptions: [TransactionLogsProcessing: [TruncateOnlyOnSuccessJob]], WinCredsId: [00000000-0000-0000-0000-000000000000], LinCredsId: [00000000-0000-0000-0000-000000000000], Old Properties: [Enabled: [True], IgnoreErrors: [False], GuestFSIndexingType: [ExceptSpecifiedFolders], IncludedIndexingFolders: [], ExcludedIndexingFolders: [%windir%; %ProgramFiles%; %ProgramFiles(x86)%; %ProgramW6432%; %TEMP%], LinGuestFSIndexingType: [ExceptSpecifiedFolders], LinIncludedIndexingFolders: [], LinExcludedIndexingFolders: [/cdrom; /dev; /media; /mnt; /proc; /tmp; /lost+found], IsFirstUsage: [True]]], 
[08.01.2024 09:40:45.322] <01> Info         Platform: [EEndPoint], UpdateVmx: [False]
[08.01.2024 09:40:45.322] <01> Info         DiskFilter: [<EpDiskFilter><BackupMode>0</BackupMode><BackupSystemState>False</BackupSystemState><BackupUserFolders>False</BackupUserFolders><ExcludeSystemState>False</ExcludeSystemState><ExcludeOneDriveFolders>False</ExcludeOneDriveFolders><Drives /><HiddenVolumesAndPartitions /><IncludeMasks /><ExcludeMasks /><BackupAllUsbDrives>False</BackupAllUsbDrives><IncludedFsItems /><UserProfilesBackupOptions ExcludeRoamingProfiles="True" ExcludeNetworkStorages="True" ExcludeTemporaryProfiles="True" ExcludeCorruptedProfiles="True" SelectedFolderTypes="0" SelectedSpecialFolders="0" /></EpDiskFilter>], ]
[08.01.2024 09:40:45.323] <01> Info             Agent policy type: Server
[08.01.2024 09:40:45.349] <01> Info         Limit for parallel tasks number is '4'
[08.01.2024 09:40:45.356] <01> Info         [CRepositoryAccessorFactory] Create, repository ID 88788f9e-d8f5-4eb4-bc4f-9b3f5403bcec, baseSession ID [ed3c2cd6-496c-466b-afd0-c568fee937ff], useSplittedRelativePathCommander=False, cloudConnectionHolder is provided: False, forceCloudGatedMode: False
[08.01.2024 09:40:45.358] <01> Info            called from Veeam.Backup.Core.AgentManagement.CEpAgentBackupJobPerformer.PrepareBackup
[08.01.2024 09:40:45.370] <01> Info         [CRepositoryAccessorFactory] Creating CBackupRepositoryAccessorImpl with sharedAgentSpec.IsEmpty = True
[08.01.2024 09:40:45.377] <01> Info         [CRepositoryAccessor] ctr, impl type [Veeam.Backup.Core.CBackupRepositoryAccessorImpl], repository ID 88788f9e-d8f5-4eb4-bc4f-9b3f5403bcec
[08.01.2024 09:40:45.391] <01> Info             Found existing backup, backup id 'd928e992-0605-468e-a971-31a8dd88a065'.
[08.01.2024 09:40:45.413] <01> Info         [from CPolicyBackupEncryptionEnabledProvider] JobId=e7ac40d1-5743-4bfe-a958-33c3c15ca4df, JobName=TJDC01, JobType=EpAgentBackup (None of the conditions are met)
[08.01.2024 09:40:45.459] <01> Info         [PolicyBackupEncryptedStateUpdater] Initializing backup encryption state (BackupId='d928e992-0605-468e-a971-31a8dd88a065', IsEncryptionEnabled=False, EncryptionState=Unencrypted, ChildStates='Unencrypted')
[08.01.2024 09:40:45.482] <01> Info         Searching for imported backups associated with backup job ( total backups number: '1').
[08.01.2024 09:40:45.512] <01> Info         
[08.01.2024 09:40:45.512] <01> Info         Building backup tasks.
[08.01.2024 09:40:45.577] <17> Info         [CEpAgentPolicyTaskScheduler] Waiting for endpoints ready to process.
[08.01.2024 09:40:45.770] <17> Info         [CEpAgentPolicyTaskScheduler] Object 524f5321-b06e-41cb-a865-cfbc13743aee contains 1 agents.
[08.01.2024 09:40:45.774] <17> Info         [CEpAgentPolicyTaskScheduler] Policy task type is ProcessAll
[08.01.2024 09:40:45.830] <17> Info         File exclude settings is ignored due to the license limitations
[08.01.2024 09:40:45.845] <17> Info         Processing task 1
[08.01.2024 09:40:45.892] <16> Info             [DB] Sync agent [63275e48-791f-4484-9b18-f7e41b25c868]: managementMode [ByVbr]
[08.01.2024 09:40:45.907] <16> Info         [CEpAgentPreparer] Agent 192.168.35.2 is ready (OK).
[08.01.2024 09:40:45.937] <16> Info         [CEpAgentPreparerDiscoveryResult] Starting discovery job for protection groups: 
[08.01.2024 09:40:45.938] <16> Info         [CEpAgentPreparer] There are no started discovery job(s)
[08.01.2024 09:40:45.948] <01> Info         [EpAgentBackupJobPerformer] Preparing agent 192.168.35.2.
[08.01.2024 09:40:45.957] <01> Info         Agent 63275e48-791f-4484-9b18-f7e41b25c868 used in job
[08.01.2024 09:40:46.161] <01> Info             [DB] Sync agent [63275e48-791f-4484-9b18-f7e41b25c868]: managementMode [ByVbr]
[08.01.2024 09:40:46.241] <01> Info         [SEpHostCredentialsFinder] Using credentials from an include container 5906fa5c-30e0-413d-98ef-9ba24c6a018a
[08.01.2024 09:40:46.446] <01> Info         [CRepositoryAccessorFactory] Create, repository ID 88788f9e-d8f5-4eb4-bc4f-9b3f5403bcec
[08.01.2024 09:40:46.447] <01> Info            called from Veeam.Backup.Core.SBackupGfsRetentionSettingsSetter.SaveGfsPoliciesAndRetentionSettingsToBackup
[08.01.2024 09:40:46.447] <01> Info         [CRepositoryAccessorFactory] Create, repository ID 88788f9e-d8f5-4eb4-bc4f-9b3f5403bcec, baseSession ID [], useSplittedRelativePathCommander=False, cloudConnectionHolder is provided: False, forceCloudGatedMode: False
[08.01.2024 09:40:46.448] <01> Info            called from Veeam.Backup.Core.CRepositoryAccessorFactory.Create
[08.01.2024 09:40:46.448] <01> Info         [CRepositoryAccessorFactory] Creating CBackupRepositoryAccessorImpl with sharedAgentSpec.IsEmpty = True
[08.01.2024 09:40:46.448] <01> Info         [CRepositoryAccessor] ctr, impl type [Veeam.Backup.Core.CBackupRepositoryAccessorImpl], repository ID 88788f9e-d8f5-4eb4-bc4f-9b3f5403bcec
[08.01.2024 09:40:46.641] <01> Info         [EpAgentManagementJobConstructor] Updated job [08c8a8b5-e63e-4797-a78e-9f14954f5a0e] flags. SqlEnabled: [False]. OracleEnabled: [False] PostgreSqlEnabled: [False]
[08.01.2024 09:40:46.668] <01> Info         [TaskBuilder] Building task blanks. JobId: 08c8a8b5-e63e-4797-a78e-9f14954f5a0e
[08.01.2024 09:40:46.849] <01> Info         [SEpAgentSourceHelper] IP addresses for EP connection info: ['192.168.35.2'].
[08.01.2024 09:40:46.909] <20> Info         [RPC] Loading options.
[08.01.2024 09:40:46.961] <01> Info         [SNetworkAddressResolver] IP address or host name [192.168.35.2] was resolved to [192.168.35.2].
[08.01.2024 09:40:46.965] <01> Info         [SNetworkAddressResolver] Using hostnames resolving policy: UseOnlyOriginalHostNames
[08.01.2024 09:40:46.967] <01> Info         [SNetworkAddressResolver] Resolved [192.168.35.2] by Kerberos preferred strategy host names and IP addresses. IPAddressKind: [All]. Result: [192.168.35.2].
[08.01.2024 09:40:47.046] <01> Info         [CProxyRpcInvoker] RpcInvoker [36702580] has been created. Host: [192.168.35.2:6160]
[08.01.2024 09:40:47.074] <20> Info         [RPC] RPC confirm session has been created. Id: {cd106534-ac8e-4ccd-ac80-1d54ee6772da}
[08.01.2024 09:40:47.088] <01> Info         [192.168.35.2] Getting 'VeeamEndpointBackupSvc' service state...
[08.01.2024 09:40:47.092] <01> Info         [192.168.35.2] The state of 'VeeamEndpointBackupSvc' service is: RUNNING
[08.01.2024 09:40:47.175] <20> Info         [RPC] Destroying RPC confirm session. Id: {cd106534-ac8e-4ccd-ac80-1d54ee6772da}
[08.01.2024 09:40:47.176] <01> Info         [CProxyRpcInvoker] RpcInvoker [36702580] was disposed
[08.01.2024 09:40:47.179] <01> Info         Connecting to VAW, ips: '192.168.35.2'
[08.01.2024 09:40:47.222] <01> Info         Trying connect to (AddressFamily: 'InterNetwork') IP addresses: ['192.168.35.2']
[08.01.2024 09:40:47.396] <01> Info         Interaction protocol version: 6.
[08.01.2024 09:40:47.416] <01> Info         Trying connect to (AddressFamily: 'InterNetwork') IP addresses: ['192.168.35.2']
[08.01.2024 09:40:47.456] <01> Info         Certificate 18f53cb8-cec2-45f9-84ea-c54ade9ec62f is used by 192.168.35.2
[08.01.2024 09:40:47.479] <01> Info         [CEpObjectsOwnershipsDbScope] [EpOwnership] Link Job [08c8a8b5-e63e-4797-a78e-9f14954f5a0e] with account [0dca4323-3e3b-404c-a91f-84f1d125ab21]
[08.01.2024 09:40:47.487] <01> Info         [CEpObjectsOwnershipsDbScope] [EpOwnership] Link Repository [88788f9e-d8f5-4eb4-bc4f-9b3f5403bcec] with account [0dca4323-3e3b-404c-a91f-84f1d125ab21]
[08.01.2024 09:40:47.491] <01> Info         [CEpObjectsOwnershipsDbScope] [EpOwnership] Link Job [e7ac40d1-5743-4bfe-a958-33c3c15ca4df] with account [0dca4323-3e3b-404c-a91f-84f1d125ab21]
[08.01.2024 09:40:47.497] <01> Info         [TaskBuilder] Getting agent info for task blank 192.168.35.2
[08.01.2024 09:40:47.508] <01> Info         [SEpAgentSourceHelper] IP addresses for EP connection info: ['192.168.35.2'].
[08.01.2024 09:40:47.580] <01> Info         [CProxyRpcInvoker] RpcInvoker [10634019] has been created. Host: [192.168.35.2:6160]
[08.01.2024 09:40:47.582] <20> Info         [RPC] RPC confirm session has been created. Id: {161d03c3-fac3-487c-867b-64d711124470}
[08.01.2024 09:40:47.584] <01> Info         [192.168.35.2] Getting 'VeeamEndpointBackupSvc' service state...
[08.01.2024 09:40:47.589] <01> Info         [192.168.35.2] The state of 'VeeamEndpointBackupSvc' service is: RUNNING
[08.01.2024 09:40:47.598] <20> Info         [RPC] Destroying RPC confirm session. Id: {161d03c3-fac3-487c-867b-64d711124470}
[08.01.2024 09:40:47.600] <01> Info         [CProxyRpcInvoker] RpcInvoker [10634019] was disposed
[08.01.2024 09:40:47.601] <01> Info         Connecting to VAW, ips: '192.168.35.2'
[08.01.2024 09:40:47.601] <01> Info         Trying connect to (AddressFamily: 'InterNetwork') IP addresses: ['192.168.35.2']
[08.01.2024 09:40:47.707] <01> Info         Interaction protocol version: 6.
[08.01.2024 09:40:47.709] <01> Info         Trying connect to (AddressFamily: 'InterNetwork') IP addresses: ['192.168.35.2']
[08.01.2024 09:40:48.151] <01> Info         Trying connect to (AddressFamily: 'InterNetwork') IP addresses: ['192.168.35.2']
[08.01.2024 09:40:48.524] <01> Info         Enumerating cluster volumes on host 192.168.35.2
[08.01.2024 09:40:48.549] <01> Info         [PolicySessionProgressController] Initializing progress
[08.01.2024 09:40:48.556] <01> Info             [PolicySessionProgressController] JobId: 08c8a8b5-e63e-4797-a78e-9f14954f5a0e. ObjName: 192.168.35.2. TotalSize: 1.9 TB. TotalUsedSize: 690.3 GB
[08.01.2024 09:40:48.564] <01> Info         [JobSession] TotalObjects '1', totalSize '1.9 TB'
[08.01.2024 09:40:48.644] <01> Info         
[08.01.2024 09:40:48.644] <01> Info         Performing backup cycles.
[08.01.2024 09:40:48.662] <01> Info         [EpAgentBackupJobPerformer] Job TJDC01 - 192.168.35.2 (08c8a8b5-e63e-4797-a78e-9f14954f5a0e) ready to start.
[08.01.2024 09:40:48.913] <13> Info         Starting job 'TJDC01 - 192.168.35.2', id '08c8a8b5-e63e-4797-a78e-9f14954f5a0e'. See log file at 'C:\ProgramData\Veeam\Backup\TJDC01\192.168.35.2\Job.192.168.35.2.log'
[08.01.2024 09:40:48.929] <01> Info         Jobs started
[08.01.2024 09:40:53.350] <20> Info         [RPC] RPC confirm session has been created. Id: {da62f230-23df-4f82-997c-9490d1f6e21a}
[08.01.2024 09:41:04.677] <20> Info         [RPC] RPC confirm session has been created. Id: {ab2a5ec1-9890-4cf8-a341-807378baee1c}
[08.01.2024 09:41:04.694] <20> Error        [RPC] An instance of the service is already running.
[08.01.2024 09:41:04.694] <20> Error        [RPC] Failed to start service 'VeeamEndpointBackupSvc'. 
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Error code: 0x00000420
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Failed to complete RPC request. Method: [StartService], In XML: [<InputArguments><ServiceName value="VeeamEndpointBackupSvc" /></InputArguments>]
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Failed to invoke RPC request
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Failed to call DoRpc. CmdName: [DoSerialRpc] inParam: [<InputArguments/>].
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Failed to call DoRpcWithBinary. Method: [DoSerialRpc], In xml: [<InputArguments/>]
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Failed to call DoRpcWithBinary
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Failed to do RPC call with serialized data, Strategy type: DefaultStrategy
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Failed to call [DoSerialRpc], session id: {ab2a5ec1-9890-4cf8-a341-807378baee1c}
[08.01.2024 09:41:04.694] <20> Error        [RPC] --tr:Failed to call [StartService]
[08.01.2024 09:41:04.723] <20> Info         [RPC] Destroying RPC confirm session. Id: {ab2a5ec1-9890-4cf8-a341-807378baee1c}
[08.01.2024 09:41:06.398] <20> Info         [RPC] RPC confirm session has been created. Id: {7a3ddda4-9fdd-479f-85c6-861bb6738e2e}
[08.01.2024 09:41:06.409] <20> Info         [RPC] Destroying RPC confirm session. Id: {7a3ddda4-9fdd-479f-85c6-861bb6738e2e}
[08.01.2024 09:45:20.938] <20> Info         [RPC] RPC confirm session has been created. Id: {9ede079d-1191-499b-8bf0-50b28579cded}
[08.01.2024 09:45:20.987] <20> Info         [RPC] Destroying RPC confirm session. Id: {9ede079d-1191-499b-8bf0-50b28579cded}
[08.01.2024 09:45:24.316] <20> Info         [RPC] Destroying RPC confirm session. Id: {da62f230-23df-4f82-997c-9490d1f6e21a}
[08.01.2024 09:45:31.277] <01> Info         [JobSession] Reloading session 'ed3c2cd6-496c-466b-afd0-c568fee937ff'
[08.01.2024 09:45:31.333] <01> Info         [CPolicyJobDeletedRetentionAlg] Deleted host retention is disabled in advanced options
[08.01.2024 09:45:31.341] <01> Info         [PolicyBackupEncryptedStateUpdater] Committing backup encryption state (BackupId='d928e992-0605-468e-a971-31a8dd88a065', EncryptionState=Unencrypted, ChildStates='Unencrypted')
[08.01.2024 09:45:31.342] <01> Info         Job has been stopped successfully. Name: [TJDC01], JobId: [e7ac40d1-5743-4bfe-a958-33c3c15ca4df]
[08.01.2024 09:45:31.345] <01> Info         Waiting for completion of management jobs.
[08.01.2024 09:45:31.351] <01> Info         Job 08c8a8b5-e63e-4797-a78e-9f14954f5a0e is completed.
[08.01.2024 09:45:31.354] <01> Info         [BackupJobPostActivity] JobSession 'ed3c2cd6-496c-466b-afd0-c568fee937ff' PostActivity: 'AskService'
[08.01.2024 09:45:31.369] <01> Info         [BackupJobPostActivity] Session result is Failed. Checking if it is last retry
[08.01.2024 09:45:31.389] <01> Info         [BackupJobPostActivity] wouldBeRetried: 'False'. Need post activity: 'True'
[08.01.2024 09:45:31.396] <01> Info         Custom command execution (post script) disabled
[08.01.2024 09:45:31.396] <01> Info         [JobSession] Completing session 'ed3c2cd6-496c-466b-afd0-c568fee937ff'.
[08.01.2024 09:45:31.396] <01> Info         [JobSession] Reloading session 'ed3c2cd6-496c-466b-afd0-c568fee937ff'
[08.01.2024 09:45:31.426] <01> Info         Backuped size: 0 B, total backuped size: 0 B
[08.01.2024 09:45:31.438] <01> Info         Job session 'ed3c2cd6-496c-466b-afd0-c568fee937ff' has been completed, status: 'Failed', '0 B' of '0 B' bytes, '1' of '0' tasks, '0' successful, '0' failed, details: '', PointIds: []
[08.01.2024 09:45:31.509] <01> Info         [BackupJobPostActivity] JobSession 'ed3c2cd6-496c-466b-afd0-c568fee937ff' PostActivity: 'AskService'
[08.01.2024 09:45:31.509] <01> Info         [BackupJobPostActivity] Session result is Failed. Checking if it is last retry
[08.01.2024 09:45:31.524] <01> Info         [BackupJobPostActivity] wouldBeRetried: 'False'. Need post activity: 'True'
[08.01.2024 09:45:31.578] <01> Info         Email notification is disabled
[08.01.2024 09:45:31.591] <01> Info         [CJobLogsManager] Cannot find quota. JobId : e7ac40d1-5743-4bfe-a958-33c3c15ca4df.
[08.01.2024 09:45:31.600] <01> Info         Cleanup cloud connection cache for job e7ac40d1-5743-4bfe-a958-33c3c15ca4df
[08.01.2024 09:45:31.621] <01> Info             Job event 'ready2finish' was created for session 'ed3c2cd6-496c-466b-afd0-c568fee937ff'. Event full name: 'Global\ready2finishvmjobevented3c2cd6-496c-466b-afd0-c568fee937ff'
[08.01.2024 09:45:31.727] <01> Info         Job [TJDC01] is ready to finish. SessionId: [ed3c2cd6-496c-466b-afd0-c568fee937ff]
[08.01.2024 09:45:31.727] <01> Info             Job event 'ready2finish' was disposed. Session: 'ed3c2cd6-496c-466b-afd0-c568fee937ff'.
[08.01.2024 09:45:31.727] <01> Info             Job event 'finished' was disposed. Session: 'ed3c2cd6-496c-466b-afd0-c568fee937ff'.
[08.01.2024 09:45:31.729] <01> Info             Unregistering TCP client channel [bs_tcp].
[08.01.2024 09:45:31.729] <01> Info             Channel successfully unregistered.
[08.01.2024 09:45:31.729] <01> Info             Unregistering TCP client channel [cat_tcp].
[08.01.2024 09:45:31.729] <01> Info             Channel successfully unregistered.
[08.01.2024 09:45:31.729] <01> Info             Job process lock was disposed.
[08.01.2024 09:45:31.729] <01> Info             Job process lock was disposed.
[08.01.2024 09:45:31.761] <08> Info         [LicenseClient] License client stopped. Stopping refresh loop...
[08.01.2024 09:45:31.763] <08> Info         [LicenseClient] License refresh loop stopped
[08.01.2024 09:45:31.765] <01> Info         Disposing SVeeamBackupService...
[08.01.2024 09:45:31.770] <01> Info             Unregistering TCP client channel [bstcp].
[08.01.2024 09:45:31.771] <01> Info             Channel successfully unregistered.
[08.01.2024 09:45:31.796] <01> Info         [Ssh] Clearing connection cache
[08.01.2024 09:45:31.798] <01> Info         [Ssh] Connection cache cleared
[08.01.2024 09:45:31.806] <01> Info         ------- Veeam Manager Stopped -------
 

Userlevel 7
Badge +17

Hi @ankitj - 

Are you using Veeam Backup Community Edition to back up a physical server, or Veeam Agent for Windows standalone on the physical server? Since you shared you’re using “v12”, I assume Community Edition of Veeam Backup (i.e. VBR)? And, you’re job is failing on the VBR server?

It looks like the Veeam Backup Endpoint service on the physical server couldn’t be started, by the looks of your error log:
[08.01.2024 09:41:04.694] <20> Error  [RPC] Failed to start service 'VeeamEndpointBackupSvc'.

Can you see if this service is running on your phys server? If not, try and start it?

Userlevel 7
Badge +20

Also based on this line it seems you might have a VBR server managing the Agent -

[08.01.2024 09:40:46.161] <01> Info             [DB] Sync agent [63275e48-791f-4484-9b18-f7e41b25c868]: managementMode [ByVbr]

 

Need to validate this as well on top of other requested things asked.

Userlevel 2

Veeam Backup Endpoint service is not there..

Userlevel 7
Badge +20

Veeam Backup Endpoint service is not there..

So, you need to deploy the Agent to the server you need to back up.

Userlevel 2

Agent already installed 

Userlevel 7
Badge +20

If the Agent is deployed, then this service must exist and if not maybe the installation of the agent was corrupted, and you need to repair/reinstall it.

 

Userlevel 4

Please right-click on the agent in the VBR console and click on "rescan". Then click directly on the server in the window (then you will see the individual steps and whether it needs to install anything). What does the rescan show? Does it recognise the agent?
Send a screenshot of this

Userlevel 2

We need to take backup of physical windows. but backup got failed. have attached all information.

 

 

Userlevel 7
Badge +20

Again what version of Veeam VBR are you running and what agent version?  Have you tried to upgrade to the latest 12.1 release?

Userlevel 4

@Chris.Childerhose  He is running VBR V12.0.0.140 P20230718

 

Userlevel 7
Badge +20

@Chris.Childerhose  He is running VBR V12.0.0.140 P20230718

 

Ah yes from the screenshot.  Sorry missed that one my apologies.  I am thinking a support case at this point like I mentioned so they can do a deep dive of the logs.

Userlevel 4

I found following links for the issue online:

Backup Fails: Agent failed to process method {DataTransfer.SyncDisk}. (veeam.com)

Al Sheppard: Veeam agent error with asynchronous read operation failed

 

I would check this, alternatively I would open a ticket.

Userlevel 7
Badge +6

I’ve seen this issue with things like low space on the server and also issues with the VSS writers.

We have also seen it with corrupt blocks on a disk causing the issue.

Userlevel 4

@Chris.Childerhose  He is running VBR V12.0.0.140 P20230718

 

Ah yes from the screenshot.  Sorry missed that one my apologies.  I am thinking a support case at this point like I mentioned so they can do a deep dive of the logs.

Totally agree with you.
It is also the case that something is transferred according to the report (presumably Recovery/EFI/.. partition) and then it fails at C:
This would indicate a corrupt file system or a corrupt disc, or something completely different. That's what support is for (perhaps more data is also supplied to it^^)

Userlevel 2

I have run CHKDSK but 0 bad sector found

 

Userlevel 7
Badge +20

At this point I suggest you open a support ticket to address the issue.

Userlevel 2

 

Now i am getting different error.

Userlevel 7
Badge +20

Please contact Support and open a ticket.  This forum is not technical support just community help.

Comment