Without seeing all the information shared with support I’d like to think they’ve gathered more information to support that claim.

Theres a few things missing here such as what the underlying storage is that provides this RAID 6 and what protocol it’s accessed via? SAS/iSCSI etc.

I’d be inclined to start here

I would love to have a screenshot of the window displaying this error for education purposes, would it be possible to send it to Rasmus -at- Veeam.com, please? I’ll make sure to anonymize any identifying factors.

it’s a local repository on the Veeam server itself. Physical server DL380 Gen10 plus with 12x 8TB SAS drives.

I forgot to ask, you checked the system log already for any notifications about refs corruption?

Maybe also use PowerShell to confirm integrity stream really is enabled for that particular backup file? There is an example here: https://learn.microsoft.com/en-us/windows-server/storage/refs/integrity-streams

just search for Get-FileIntegrity

Yes, no corruption reported.

Did that, integrity streams are enabled and enforced.

On it’s way.

Unfortunately the support case was closed without finding the solution. Support insisted that it was file system corruption and I’m convinced that can’t be the case since integrity streams are enabled on the backup files and no corruption is being reported in the eventlogs. I can copy the backup files fine, even the one which has the crc error. When there’s an integrity stream violation, Windows would throw an error during this copy operation. So to me this looks like the crc error occurred during the creation of the backup file, but I wasn’t able to convince support on that and they insisted it was file system corruption. Hence, I decided to close the case since it was leading nowhere,

Have you tried to escalate your support case? If you disagree with the solution, support should continue analyzing the issue. Although I wouldn't know what issue from Veeam's side would cause a single corrupted backup file.

I've seen cases where a bad SFP module (FC) caused corrupted backup files during their creation. So maybe you should check the whole environment.

And maybe you could create a seperate job for this VM with compression/deduplication disabled to rule out any processor or memory issues?

no I didn’t. 3 engineers picked up the case after each other and they all went down the same road. So I was quite done with it. We have 23 backup jobs and 45 copy jobs running and this single job had one crc error. The server (Proliant DL380 gen10 plus) has ECC memory, so a RAM issue could also be ruled out I guess. So I really doubt a hardware issue could be the problem or else we would have seen it on much more jobs. Health checks on all jobs also pass without issues. After creating a new active full I haven’t had the issue again so I treat it as a one time occurrence for now. I’m glad Veeam does checksum checks again when reading the files, so that’s a positive outcome. ;-)

My point merely is that support pointed right to filesystem corruption, because that’s the most probable cause of this error. However, they refused to look at other potential causes even though I provided evidence that rules out files system corruption.

Ok, so it happend only once. I thought at first that you would see corruptions regularly with that VM. Probably analyzing isn't easy if the issue itself isn't reproducible.

ReFS would only let you know if corruptions occur after creation of a backup file. But if it happens during the creation, it wouldn't notice anything.

But you're right, good to see that Veeam will discover such corruptions and not silently copy them over.