Ever had your backups fail? I sure have! But it’s all part of the fun & games of learning exactly how these systems work.
Recently I spun up another instance of Veeam Backup for Microsoft Azure, and as part of my security mindset, I configured some elements of hardening to my infrastructure. Well, I missed a setting that I’m going to share with you today.
Worker Job Completed with Error
So, I had Veeam Backup for Microsoft Azure working, my daily snapshot job was also running fine, but it was the daily backup that wasn’t working. Some of the key terms I could pick out from the logs were ‘Worker: <Resource Name> (IP Address:Port): job completed with error’, ‘Error: This request is not authorised to perform this operation’, ‘Status 403’, and ‘ErrorCode AuthorizationFailure’ were the error messages I was being greeted with.
Dissecting the Errors
The key hint here was the job task ‘Backing up <VM Name> to Backup Repository. Followed by the worker not being authorised.
Within Veeam Backup for Microsoft Azure, backups are stored on storage accounts, so I started here. I’d configured my storage account to only be accessible from my VNETs and from the public IP address of my Veeam Backup for Microsoft Azure virtual machine, so I was quite confident I’d done everything right, but this was where I was mistaken.
The Root Cause: Worker Permissions
Unfortunately, whilst I had set the relevant permissions, my worker VM needed to interact with this storage account, but it didn’t exist at runtime. When Veeam Backup for Microsoft Azure creates the worker VMs, it creates appropriate VNETs for them within their region, and because I’d configured my security permissions prior to the creation of the worker VM, I’d never included the worker VM’s VNET into my networking security rules for access to my storage account!
Once this rule was added, it worked!