• EN
Solved

Backup Repository Concurrent Tasks

  • 25 October 2022
  • 8 comments
  • 5135 views
fspadaro
Userlevel 3
Badge +1

Hello everyone, it’s correct that concurrent tasks limit on the following repositories depend on the hardware (CPU/RAM) of the gateway server (transport server):

  • DataDomain connected in FC to gateway servers with 16 cores and 32GB RAM
  • Immutable on Linux servers with 16 cores and 32GB RAM

Concurrent tasks are 32 or 16VMs as reported in the following documentation:

Limitation of Concurrent Tasks - User Guide for VMware vSphere (veeam.com)

NOTE

Task limits set for backup infrastructure components influence the job performance. For example, you add a VM with 4 disks to a job and assign a backup proxy that can process maximum 2 tasks concurrently for the job. In this case, Veeam Backup & Replication will create 4 tasks (1 task per each VM disk) and start processing 2 tasks in parallel. The other 2 tasks will be pending.

 

Thanks

Federico

 

icon

Best answer by DChiavari 31 October 2022, 11:00

View original
Federico S. - VMCA2022 | VMCE2021 | VCAP-DCV/VCP-DCV | VCAP-NV/VCP-NV

8 comments

fspadaro
Userlevel 3
Badge +1

Sorry I’ve create a topic like “question” but I forgot the question mark. Could Anyone confirm me the recommended maximum concurrent tasks is like above?

 

Thanks

Federico S. - VMCA2022 | VMCE2021 | VCAP-DCV/VCP-DCV | VCAP-NV/VCP-NV
Stabz
Userlevel 7
Badge +7

Hello,
Yes concurrents task are limited by CPU/RAM.

You have to take in consideration the limitations on the proxy side and repository side

Task Limitation for Backup Proxies:
1CPU= 1 task = 1 vDisk

You could find here example how to size your proxy:
https://bp.veeam.com/vbr/2_Design_Structures/D_Veeam_Components/D_backup_proxies/vmware_proxies.html

Task Limitation for Backup Repositories:
That depends how you set the configuration of your repo, in "Per-VM mode" 1 task = 1 VDisk
You could find here example how to size your repo:
https://bp.veeam.com/vbr/2_Design_Structures/D_Veeam_Components/D_backup_repositories/

For Data Domain you need to deploy a Gateway Server, I never installed this kind of appliance, but you have to follow the best practices guides to set the good configuration. Especially to have the good block size, the number of parallels task etc...

Philippe DUPUIS |Veeam Certified Engineer | https://original-network.com/
Scott
Userlevel 7
Badge +8

Veeam best practices and documents are the best place to look for this. 1 task = 1 CPU

 

I believe that is physical CPU and not thread, but I could be wrong. However, i usually go by thread/core and overprovision a tad as my proxies are also my repos and currently used for tape. I also have a few SANS configured for different task. so lets say I give each SAN 10 task limits, and I have 3 SANS, that would be 30 cores.  However, I’m never using all 3 at once.  in my case, each task would take up 1 cpu for each disk being backed up, and 1 for the repo. if I'm running copy jobs those take up tasks too.

 

It really comes down to your infrastructure, and how you configured your jobs / repos / proxies.

 

On that note, I have 40+ Cores per proxy and 256GB of memory and I hammer them without performance issues. In most cases your target SAN will end up being the bottleneck if you have a decent production SAN and good proxies I find.  I have 16GB and 32GB fiber as well so the chances of that becoming a bottle neck are low. If you are on 1GB network that will most likely be your limit too. 

 

 

HangTen416
Userlevel 7
Badge +10

For Data Domain repository, follow the BP found here: Data Domain - Veeam Backup & Replication Best Practice Guide especially the recommendation to put the proxy and gateway roles on the same server(s).

VMCE2024 | VMCA2022 | Veeam Legend
DChiavari
Userlevel 6
Badge +7

Ciao Federico 😊

The “hard” limit for concurrent tasks is the number you set at the Repository level - after reaching that number, additional tasks will be pending / waiting for resources. In the job statistics you can clearly see what component was not available and for how long.

Of course, there are best practices about setting a “good” number for concurrent tasks based on the machine’s resources, as others have shared already. This LINK shared by @Stabz is a good starting point.

In general, Proxies are more CPU-intensive while Repositories are more RAM-hungry.

For sizing, you can consider Gateway servers exactly as Repositories, because basically they perform the very same tasks.

As far as parallelism works, be sure to size the environment to have the right amount of task slots based on your desired concurrency. Keep in mind that for Agent backups, the “Proxy” role is carried out by the Agent itself. See example below:

 

Danilo Chiavari - Veeam Presales Manager, Italy - www.danilochiavari.com
fspadaro
Userlevel 3
Badge +1

Thanks a lot to all.

I understand that there isn’t a correct answer but there are multiple variables that need to be analyzed starting the configuration as Veeam BP recommends 

Federico S. - VMCA2022 | VMCE2021 | VCAP-DCV/VCP-DCV | VCAP-NV/VCP-NV
vmJoe
Userlevel 7
Badge +7

The best practices guide for the concurrent tasks.  Th eBP guide wills its I/3rd of the total amount of proxy tasks (cores) send data to the repo.  Remember the gateway needs to sized like a repository sever so make sure you follow the RAM and CPU recommendations.

 

On more thing you will need to consider when sizing the repo with the required tasks.  Will you be doing any SOBR offloads to object storage, or backup copy jobs? If so then make sure you have some extra task slots (CPU) available in your gateway server so the offload can keep running when the backup runs. Th backup job has priority over offload jobs so it will take tasks assign to offload to complete a backup job if there is not enough proxy tasks assign to the gateway.

Joe Gremillion, VMCA, TOGAF, VeeaMVP
fspadaro
Userlevel 3
Badge +1

The best practices guide for the concurrent tasks.  Th eBP guide wills its I/3rd of the total amount of proxy tasks (cores) send data to the repo.  Remember the gateway needs to sized like a repository sever so make sure you follow the RAM and CPU recommendations.

 

On more thing you will need to consider when sizing the repo with the required tasks.  Will you be doing any SOBR offloads to object storage, or backup copy jobs? If so then make sure you have some extra task slots (CPU) available in your gateway server so the offload can keep running when the backup runs. Th backup job has priority over offload jobs so it will take tasks assign to offload to complete a backup job if there is not enough proxy tasks assign to the gateway.

Thanks vmJoe after first backup on immutable repository I must create two backup copies to two different destinations.

As you say I need to calculate extra task slots available on Gateway Server (Linux server direct attach on storage immutable) to manage these secondary copies.

Thanks 

Federico S. - VMCA2022 | VMCE2021 | VCAP-DCV/VCP-DCV | VCAP-NV/VCP-NV

Comment


Terms & Conditions