Question

Veeam backup for Azure failed to backup a VM to Azure repository


Userlevel 2
  • Not a newbie anymore
  • 6 comments

Hi, we have Veeam backup for Azure all setup and running and its been a couple of months. We have private network deployment with private endpoints in our environment with premium service bus and all subnets are allowed to access this service bus. We have 5 backup policies in total, 3 of them have been running fine but 2 of the jobs (with our file servers), the snapshots are running fine but the backup policy is getting failed after 4/5 hours of the session run with an error :

Failed to perform backup of vm1 to Production Backups Repo 1. Worker VBA-5bf0973a-8e2c-4dda-ba63-28814324c1bb: getting of the job status failed (trace ID: b8b3fb4e-c7f0-4f86-aa22-4f2122f45b00)

We have noticed that when the backup policy session starts, it backs up the data all the way to 60% percent and then it stuck for a very long time before erroring out this message. We have raised a support case with Veeam and they said they will need to check with their R&D team, as we have the same settings for all jobs and getting this error on only 2. We would appreciate any advice / suggestions? Many thanks


12 comments

Userlevel 7
Badge +20

It is probably best to continue working with Support on this one as the error is vague and they will be able to look at your logs to determine the root cause.

Best of luck getting it resolved.

Userlevel 2

Thanks Chris, yes we have been working with RND team since last week, lets see if we can find out the root cause.

Userlevel 7
Badge +7

@gabbas You can download the logs and have a look at them. You may find some root causes if it’s easy to fix it.

But as @Chris.Childerhose said if the issues are complicated you are better to continue working with Veeam support. 

https://helpcenter.veeam.com/docs/vbazure/guide/support_information.html?ver=5a

 

Userlevel 2

@gabbas You can download the logs and have a look at them. You may find some root causes if it’s easy to fix it.

But as @Chris.Childerhose said if the issues are complicated you are better to continue working with Veeam support. 

https://helpcenter.veeam.com/docs/vbazure/guide/support_information.html?ver=5a

 

Thanks, downloaded the logs and reviewed and found this error:

Error: Veeam.Azure.Workers.Contracts.Exceptions.WorkerNotRespondingException: Failed to connect to worker with ID VBA-4308e41b-9ed5-4fe9-921b-1e3a26452016
 ---> Microsoft.Azure.ServiceBus.UnauthorizedException: Put token failed. status-code: 401, status-description: Ip has been prevented to connect to the endpoint.

 

Double checked, all required vNets are added to the FW exception of the service bus and all other jobs are working as expected so this is very odd. 

Userlevel 7
Badge +20

@gabbas You can download the logs and have a look at them. You may find some root causes if it’s easy to fix it.

But as @Chris.Childerhose said if the issues are complicated you are better to continue working with Veeam support. 

https://helpcenter.veeam.com/docs/vbazure/guide/support_information.html?ver=5a

 

Thanks, downloaded the logs and reviewed and found this error:

Error: Veeam.Azure.Workers.Contracts.Exceptions.WorkerNotRespondingException: Failed to connect to worker with ID VBA-4308e41b-9ed5-4fe9-921b-1e3a26452016
 ---> Microsoft.Azure.ServiceBus.UnauthorizedException: Put token failed. status-code: 401, status-description: Ip has been prevented to connect to the endpoint.

 

Double checked, all required vNets are added to the FW exception of the service bus and all other jobs are working as expected so this is very odd. 

Seems like Veeam is having a hard time finding the appliance with the VBA- ID.  Might start there with support.

Userlevel 7
Badge +7

Try to review the worker settings, you may find some clues.

https://helpcenter.veeam.com/docs/vbazure/guide/worker_config_manage.html?ver=5a

 

Userlevel 2

Just a quick question while we are still waiting to hear back from Veeam RND team, we have all our workload in one Azure region. The storage accounts that we have configured the backup repo in Veeam, have private endpoints and with public access disabled (only allowing Veeam worker vnet in the allowed network list). If we turn off the “private network deployment” option in Veeam, and try to run the jobs again, we have noticed the backup gets completed without any issue. My question would be if we keep that option (private network deployment) disabled, will that expose our data to public while we have our storage accounts and repos are configured with private endpoints? As per Veeam suggestion, private endpoints are ideal if your workload is in multiple regions so with only one region, I am just thinking what additional benefit this could bring us with private deployment?

Userlevel 2

Just a quick question while we are still waiting to hear back from Veeam RND team, we have all our workload in one Azure region. The storage accounts that we have configured the backup repo in Veeam, have private endpoints and with public access disabled (only allowing Veeam worker vnet in the allowed network list). If we turn off the “private network deployment” option in Veeam, and try to run the jobs again, we have noticed the backup gets completed without any issue. My question would be if we keep that option (private network deployment) disabled, will that expose our data to public while we have our storage accounts and repos are configured with private endpoints? As per Veeam suggestion, private endpoints are ideal if your workload is in multiple regions so with only one region, I am just thinking what additional benefit this could bring us with private deployment?

@Chris.Childerhose Any advice on this please?

Userlevel 7
Badge +20

Just a quick question while we are still waiting to hear back from Veeam RND team, we have all our workload in one Azure region. The storage accounts that we have configured the backup repo in Veeam, have private endpoints and with public access disabled (only allowing Veeam worker vnet in the allowed network list). If we turn off the “private network deployment” option in Veeam, and try to run the jobs again, we have noticed the backup gets completed without any issue. My question would be if we keep that option (private network deployment) disabled, will that expose our data to public while we have our storage accounts and repos are configured with private endpoints? As per Veeam suggestion, private endpoints are ideal if your workload is in multiple regions so with only one region, I am just thinking what additional benefit this could bring us with private deployment?

@Chris.Childerhose Any advice on this please?

Unsure what advice to give here as I don’t use Azure much.  If this is working with it off that is good but cannot say for sure about data being exposed.  If the storage is on the private network but there are ways in to the VBR server via a public address someone could always get to the data in some fashion.

Might be best to ask Support about this while waiting on RnD.

Userlevel 2

Just a quick question while we are still waiting to hear back from Veeam RND team, we have all our workload in one Azure region. The storage accounts that we have configured the backup repo in Veeam, have private endpoints and with public access disabled (only allowing Veeam worker vnet in the allowed network list). If we turn off the “private network deployment” option in Veeam, and try to run the jobs again, we have noticed the backup gets completed without any issue. My question would be if we keep that option (private network deployment) disabled, will that expose our data to public while we have our storage accounts and repos are configured with private endpoints? As per Veeam suggestion, private endpoints are ideal if your workload is in multiple regions so with only one region, I am just thinking what additional benefit this could bring us with private deployment?

@Chris.Childerhose Any advice on this please?

Unsure what advice to give here as I don’t use Azure much.  If this is working with it off that is good but cannot say for sure about data being exposed.  If the storage is on the private network but there are ways in to the VBR server via a public address someone could always get to the data in some fashion.

Might be best to ask Support about this while waiting on RnD.

Thanks @Chris.Childerhose appreciated. I will ask the support team. 

Userlevel 7
Badge +20

Just a quick question while we are still waiting to hear back from Veeam RND team, we have all our workload in one Azure region. The storage accounts that we have configured the backup repo in Veeam, have private endpoints and with public access disabled (only allowing Veeam worker vnet in the allowed network list). If we turn off the “private network deployment” option in Veeam, and try to run the jobs again, we have noticed the backup gets completed without any issue. My question would be if we keep that option (private network deployment) disabled, will that expose our data to public while we have our storage accounts and repos are configured with private endpoints? As per Veeam suggestion, private endpoints are ideal if your workload is in multiple regions so with only one region, I am just thinking what additional benefit this could bring us with private deployment?

@Chris.Childerhose Any advice on this please?

Unsure what advice to give here as I don’t use Azure much.  If this is working with it off that is good but cannot say for sure about data being exposed.  If the storage is on the private network but there are ways in to the VBR server via a public address someone could always get to the data in some fashion.

Might be best to ask Support about this while waiting on RnD.

Thanks @Chris.Childerhose appreciated. I will ask the support team. 

Not a problem wish I could help more but learning Azure stuff now. 😂

Userlevel 7
Badge +7

@gabbas You may have already seen this but if not,  you may need to make some modifications as detailed here https://helpcenter.veeam.com/docs/vbazure/guide/app_private_network.html?ver=5a

Comment