• EN
Solved

Veeam Office 365 backup painfully slow!

D
Userlevel 2

Hi, I have downloaded a trial of Veeam Office 365 backup after hearing how good it was in several places as we currently use another product but it's not the best in terms of features. Initial setup was simple enough but when it came to running the backup job, the performance was dreadfully slow. Averaging between 500-800kbps, it will take months to backup at this rate! Bandwidth is not an issue, we have a leased line and easily get 200meg down. 
Surely something must be wrong? I have read about auxiliary accounts but that option is greyed out for me...? I have the latest version, downloaded only last week. A bit disappointing since the rest of it seems to be pretty sweet.

Any help is appreciated, thank you.

icon

Best answer by Chris.Childerhose 11 January 2022, 20:40

View original

21 comments

Chris.Childerhose
Userlevel 7
Badge +20

Seeing that rate it will definitely be throttling on the MS side playing a part.   You need to be able to set up the Aux accounts to use for backups which help improve on speed.  You may need to work with the person that is the O365 Admin to do this or have the account you are working with changed to Admin in the system as it sounds like it might not be.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
Chris.Childerhose
Userlevel 7
Badge +20

You can also check this page about the Aux accounts as it has links to other pages including how to use PowerShell to add the accounts - Office 365 Backup: Auxiliary accounts, cloud to cloud (veeam.com)

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
Chris.Childerhose
Userlevel 7
Badge +20

Also keep in mind the throttling is typically on SharePoint and OneDrive normally and typically not mailboxes.  So job configuration may need to be changed with separate jobs for each.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
regnor
Userlevel 7
Badge +14

Just in case you're experiencing throttling during the backup of mailboxes, this post describes how to temporarily disable it:

 

Max M. | Systems Engineer @Veeam | VMCE/VMCA | former Legend & Vanguard
MicoolPaul
Userlevel 7
Badge +20

Just in case you're experiencing throttling during the backup of mailboxes, this post describes how to temporarily disable it:

 

This is great if you’re being throttled by Exchange. If your bottleneck is OneDrive/SharePoint then auxiliary accounts will help.

 

Can you also confirm the spec of your server? Cores + RAM. Could equally be issues with memory pressure. There’s a room for sizing estimation for resources that would be good to run too if you haven’t yet. How many users and SharePoint sites are you trying to protect? Any ideas of total data size?

Michael Paul - #VeeamLegend | #VeeamVanguard | Veeam Certified Architect | VMware vExpert | https://micoolpaul.com | Twitter: @MicoolPaul | Mastodon: @micoolpaul@masto.nu | Bluesky: @micoolpaul.bsky.social
D
Userlevel 2

Wow, loads of replies in very little time! This is a great community!

I’ll reply to each separately

D
Userlevel 2

Seeing that rate it will definitely be throttling on the MS side playing a part.   You need to be able to set up the Aux accounts to use for backups which help improve on speed.  You may need to work with the person that is the O365 Admin to do this or have the account you are working with changed to Admin in the system as it sounds like it might not be.

 Like I said, the “manage backup accounts” option in the right click menu for the org is greyed out (no idea why...) I am the 365 admin so I could create as many accounts as needed. I created an account specially for the backup which is currently a global admin and given the migrator role which has applicationimpersonation which I read somewhere it was a good idea to have on. 

D
Userlevel 2

Also keep in mind the throttling is typically on SharePoint and OneDrive normally and typically not mailboxes.  So job configuration may need to be changed with separate jobs for each.


I’m only backing up accounts, not Sharepoint. I did a test by backing up accounts that had large mailboxes and very little onedrive data and other accounts which had lots of onedrive data and almost no mail, and they were both as slow as could be. I also did a test account which had almost nothing, like 5 emails and 2 files and that took 10 minutes to back up (??)

D
Userlevel 2

Just in case you're experiencing throttling during the backup of mailboxes, this post describes how to temporarily disable it:

 

Never knew this existed, this is brilliant thank you! I’ll test tomorrow morning and see what happens, although I can’t see myself having to turn throttling off before every backup, but it’s ideal to troubleshoot.

randyweis
Userlevel 5
Badge +4

The Backup Service Accounts only apply to the OD and SPO jobs. You can increase threads for the email job, but that doesn’t always allow for increased throughput.

Regnor’s suggestion should be tried first.

The other thing to do is to strategize backups. Don’t try to back up everything at once. If you don’t want to break the job up into groups, you can go to exclusion and exclude nearly every group. As each group gets the first backup completed, remove another group (or 2 or 3) from the exclusion list and let those complete. Rinse and repeat.

Throttling is out of Veeam’s control, but double check your networking to be sure there isn’t anything misconfigured. Then turn off throttling and use exclusions.

 

Finally, please read this document from start to finish. This has EVERYTHING you need.

https://www.veeam.com/wp-guide-veeam-backup-microsoft-office-365-v5.html?wpty

Randy Weis, VMCE, VMCA
D
Userlevel 2

I’m afraid turning off throttling made no difference at all…… Not sure if it’s because I’m doing a test during office hours but the speeds are now 100-500kbps, sometimes dipping into double digits!

 


I just tested backing up only a couple of user accounts with a relatively average amount of mail (5GB max) and empty OneDrives. 

I can’t test the additional accounts though (although it sounds from what was said before that it won’t make a difference):
 

 

D
Userlevel 2

Can you also confirm the spec of your server? Cores + RAM. Could equally be issues with memory pressure. There’s a room for sizing estimation for resources that would be good to run too if you haven’t yet. How many users and SharePoint sites are you trying to protect? Any ideas of total data size?

As this is a test, the backup environment is as good as it can be. I’m backing up onto a bare metal Gen9 ProLiant - 8 Cores, 128GB. The backup proxy and repo are local on the same box so nothing fancy and surely optimal for speed (I assume)

I’ve reduced my data size for the test to only 5 accounts averaging 5GB mailboxes and empty OneDrives. They took overnight to backup, so god knows how I’m gonna backup 500 accounts with larger mailboxes and 200GB OneDrives. I’m skipping Sharepoint for now.

Chris.Childerhose
Userlevel 7
Badge +20

The server has good specs. The repo drive what is the file system format?  NTFS or ReFS?  Hopefully NTFS because of the database for O365 being a Jet DB does not work well on ReFS.

Also the internet connection you have what is the speed there?  Typically you don't run backups during business hours but you indicated they worked overnight too.

Might be worthwhile opening a case with support and uploading the logs to get a better idea.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
D
Userlevel 2

The server has good specs. The repo drive what is the file system format?  NTFS or ReFS?  Hopefully NTFS because of the database for O365 being a Jet DB does not work well on ReFS.

Also the internet connection you have what is the speed there?  Typically you don't run backups during business hours but you indicated they worked overnight too.

Might be worthwhile opening a case with support and uploading the logs to get a better idea.

Yep, NTFS

Internet connection gets a solid 200Meg down (it’s a leased line) I just synced a Sharepoint library of about 50GB on a user’s machine and it took about half an hour so the internet connection is definitely not the bottleneck.

Chris.Childerhose
Userlevel 7
Badge +20

Sounds like it is not but better to ask. 😃

At this point I think a support case would help to see if Veeam can determine the bottleneck.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
D
Userlevel 2

I’ve disabled Powershell throttling and my speeds doubled (still very slow)

 


 

 

 

I have logged a support case anyway to see what else is going on

regnor
Userlevel 7
Badge +14

Please keep us updated on your support case. The cause of your performance issues will be interesting, as we've already ruled out many possibilities.

Max M. | Systems Engineer @Veeam | VMCE/VMCA | former Legend & Vanguard
marcofabbri
Userlevel 7
Badge +13

Any update @drkmccy ?

That’s an interesting case to solve.

Backups are like pizza, I love them. | Linkedin: @marco-fabbri-it
D
Userlevel 2

Any update @drkmccy ?

That’s an interesting case to solve.

My case was closed without resolution. Something about being too busy.

Mildur
Userlevel 7
Badge +12

My case was closed without resolution. Something about being too busy.

I‘m sorry to hear that.

This can happen with the trial license. Support is on best effort. Paying customers are prioritized over the trial (or free) product users. You can try to open a new support case. 

Product Management Analyst @ Veeam Software
M
Userlevel 2

I'm experiencing similar problems with M365 backups, where the speed is around 14-16MB/s when only backing up mailboxes. The bottleneck seems to be the source according to the job analysis. Is it normal to have this throughput when backing up mailboxes from M365? Can disabling EWS and Powershell throttling improve the speed in this situation? What speeds are others achieving in this regard?

Comment


Terms & Conditions