In The Lab With: Veeam Backup for Microsoft 365 v6 - What's new? What's fixed? What should I know?


Userlevel 7
Badge +20

 

 

Today Veeam have launched the new Veeam Backup for Office Microsoft 365 v6! That’s right, it’s had a slight name change to align with Microsoft’s rebrand of Office 365 to Microsoft 365. Excited and want to check it out? Download it here. Veeam have also updated their Helpcenter ready to explain all the new features. As always, Veeam have accompanied this release with a set of release notes, available here.

In this blog post I’ll be going over the key new features and also any key insights to be aware of as part of deploying Veeam Backup for Microsoft 365 v6 or upgrading to v6 from an earlier version.

 

Key Information

 

Veeam Backup for Microsoft 365 Server:

Straight away, I want to highlight that the CPU requirements have changed. Veeam are now recommending 8 cores as the minimum for v6, up from 4 cores in v5.

It’s the same story for RAM, v6 now requires 16GB as a minimum, up from 8GB. There is a discrepancy between the release notes and the Helpcenter documentation with regards to VMs with dynamic memory allocation. The release notes state 8GB RAM is the minimum, however the Helpcenter documentation states 16GB, I would recommend sticking to the higher figure and configuring a minimum of 16GB.

From an Operating System support perspective, v6 brings support to running on Windows 11 and Windows Server 2022.

On a positive note, Application Aware Processing of the v6 server is still supported by the same versions of Veeam Agent and Veeam Backup & Replication as v5 (v4 onwards and v10 onwards respectively).

 

Veeam Backup for Microsoft 365 Proxy:

Unlike the main server components, there are no changes required to the proxy to meet minimum requirements, though as always, more performance can be gained by allocating additional resources.

The proxy role is now supported on Windows 11 and Windows Server 2022 Operating Systems.

 

Backup Targets:

Veeam haven’t changed the requirements for Backup Targets, of note is that SMB 3.0 share support is still experimental, if you were hoping for full support in this version, you’re out of luck.

We do however now see support for “Backup Copy Targets”, I’ll explore this further on but we can now leverage Azure Archive Storage, Amazon S3 Glacier and Amazon S3 Glacier Deep Archive.

 

Supported Products:

Whilst not mentioned within the release notes, on the Veeam Helpcenter (and within some specific Veeam Explorer release note comments) it is documented that Veeam Backup for Microsoft 365 v6 brings support to Microsoft SharePoint Subscription Edition.

 

Upgrading:

When upgrading an existing deployment of Veeam Backup for Office 365, version 4.0 and newer are supported for a direct upgrade to Veeam Backup for Microsoft 365 v6, excluding upgrading from beta versions.

Veeam have provided the following key points regarding upgrading:

  • Windows 7 SP1, Windows 8.x and Windows Server 2008 R2 SP1 are no longer supported.
  • Backup job & Global config settings will all be preserved.
  • Any modifications made to the Config.xml file manually will be lost.
  • All backup jobs, scheduled or not, will be stopped during the upgrade process.
  • Any backup jobs that were created by v2 or earlier and are marked as “Out of Date” WILL be deleted after upgrading. Upgrade/recreate these jobs prior to upgrading to v6 if they are still required.
  • After upgrading to v6, Exchange Online backups may fail if using modern app-only authentication or modern authentication with legacy protocols with application secret. The error is “Unauthorized (401)”, to resolve this, the Azure AD application requires additional permissions, specifically the “full_access_as_app” API permission. Veeam have detailed this in KB4169.
  • Veeam can’t automatically update remote REST services or Remote Veeam Explorers.
  • A workgroup-based remote backup proxy upgrade will fail unless proxy host administrator account details are provided.
  • After upgrading, REST clients will need to re-authenticate with user credentials instead of using a refresh token.
  • If a machine with Veeam components has been rolled back after an automatic update is installed, Veeam will not attempt to automatically update the components again.

Veeam have a documented approach as well within the release notes to detail how to install Veeam Backup for Microsoft 365 on a machine that had the beta version. However as there is no compatibility between the backup repositories or object storage repositories used by the pre-release builds, I would recommend you simply start again. But the notes are available if this isn’t an option.

If updating from v4.0 to v6, once the update has completed, you’ll notice the following:

  • All components and jobs will be marked as “Out of Date”, these components will need to be upgraded manually from the console.

If upgrading from v4.0, v4a or v4b to v6, once the update has completed, you’ll notice the following:

  • All backup jobs that contain Microsoft 365 group objects will be marked as “Out of Date”. These jobs need to be upgraded manually from the console.

 

New Features

With any major release of a Veeam product, I always tend to ask, what’s new? So, here’s the key highlights from this release:

 

Backup Copy:

Veeam have expanded their support for Veeam Backup for Microsoft 365 in version 6 by supporting a backup copy option for Azure & AWS object storage-based backups. What’s most impressive about this feature is that there’s no requirement to be using Azure or AWS for your primary backup, which was a limitation I was expecting based on how Veeam Backup & Replication utilise object storage in their Scale-out Backup Repositories.

The only requirement to leverage Backup Copy is the use of object storage as your primary backup type, as long as you’re using a supported object storage platform, you’ll be fine to use Backup Copy.

A key point to highlight here is that Veeam doesn’t just copy the blobs from the normal backup repository to the backup copy repository identically. Instead, Veeam creates larger blob files for the backup copies by repacking the backed-up data. This can be performed by either the Veeam Backup for Microsoft 365 backup proxy server, or by the deployment of an auxiliary archiver appliance. I’ll likely cover this further in a separate blog post when I have some useful metrics on how this impacts API calls etc.

Remember also that you can’t retrospectively attach object storage to a repository, so you’ll be creating a new repository with archive storage attached as your backup copy target.

Handy tip: If you missed out on deploying to object storage when you first set up this product, you can use the Move-VBOEntityData cmdlet to migrate from traditional block storage to object. But remember, you can’t migrate back! You’d need to create a whole new backup for that task. More information on the cmdlet is available here.

 

Enhanced Multi-Geo Support:

An unfortunate downside to Microsoft 365 being a SaaS product is you have no control over when new features become available, and if you’ve got siloed teams, they can implement features that you can’t support yet. Multi-Geo Support was one of these breaking features, SharePoint sites weren’t supported if they were hosted in a different location to the tenant’s primary location. Thankfully after much waiting, this feature is now available! Please check the known issues section though as there is still currently one known issue with Multi-Geo.

 

Veeam Service Provider Console Integration:

If you’re offering a managed Veeam Backup solution for Microsoft 365 and using the Service Provider console, you’ll want to upgrade to v6. Veeam Backup for Microsoft 365 v6 supports integration with the VSPC. The key benefits to this are ease of license monitoring & management, alongside a single pane of glass monitoring of the state of the Veeam Backup for Microsoft 365 services. More information is available here.

 

Self Service Portal:

Anyone that knows and loves the Veeam Backup Enterprise Manager self-service portal will likely have been asking for this feature to enable self-service recovery of data, reducing the burden on your data protection/IT teams.

Veeam have provided a Self-Service Portal with Veeam Backup for Microsoft 365 v6. This portal has RBAC support, split into three categories.

  • Veeam Backup for Microsoft 365 Administrators, who in addition to maintaining the application’s infrastructure and settings, can configure the Restore Portal settings and assign permissions to users who will act as restore operators.
  • Restore operators, who can restore data for other users.
  • End users, who can only perform a self-service restore of their own backed-up data.

As this is an initial release there are some key limitations to the product, I’ll list this later on within the “Known Issues” section.

 

Enhanced SharePoint Recoverability:

New to v6 is the ability to restore SharePoint sites to new locations, but even more amazing is that you can use Veeam to restore SharePoint sites to different organizations!

 

NEW 21/03/2022 – Automatic Updates:

That’s right, Veeam have included support for automatic updates within the latest Veeam Backup for Microsoft 365 v6 release! There are some conditions to this thankfully which primarily aim to prevent frustration and ensure data integrity. Namely updates will only occur when all of the following criteria is met:

  • No restore sessions are currently taking place.
  • All instances of the Veeam Backup for Microsoft 365 Console and PowerShell console are closed.
  • Backup, backup copy, data management and retrieval jobs aren’t running on the local backup proxy.

There are also some limitations to the automatic patching such as Veeam won’t automatically patch the Veeam Explorers if Veeam Backup for Microsoft 365 v6 co-exists on a server alongside Veeam Backup & Replication for tenants. Veeam Backup for Microsoft 365 v6 also won’t automatically update the REST API components when deployed to a remote server.

Veeam handles this via two upgrade windows, the first focusing on a predicted low period of utilisation for the backup server itself, this will patch the core Veeam Backup for Microsoft 365 v6 application, then patching the Veeam Explorers, console and PowerShell components after the core application has been successfully patched. The second upgrade window will focus on low utilisation of remote proxy servers, to upgrade them at an appropriate time.

If desired, it is possible to disable the automatic patching feature.

For the full information on this new feature, read the helpcenter page here.

 

Known Issues/Expected Behaviours/Limitations

 

General Known Issues & Limitations:

Veeam supply a list of all known limitations and issues within their release notes, if you’re upgrading, you’ll be most interested to know what is new or different, so I’ve listed these below:

  • New – Microsoft Azure Germany region is provided with experimental support.
  • Key remaining issue – If you upgrade to Veeam Backup for Microsoft 365 v6 and have updated the Veeam Explorers on the same server that Veeam Backup & Replication v10 is installed, you’ll still have compatibility issues if you upgrade to v10a. The workaround is the same, uninstall the Veeam Explorers before upgrading to v10a, then re-install the Veeam Backup for Microsoft 365 v6 Explorers after the upgrade.
  • Key remaining issue – The previous limitations for Modern Authentication only processing are still valid, see the KB here.
  • New – Azure AD application used for the Restore Portal cannot be created automatically on a host where only the Veeam Backup for Microsoft 365 remote REST service is installed.
  • New – On the Restore List tab, item’s restore status may be displayed incorrectly if multiple items of the same data type (Exchange, SharePoint or OneDrive) have been restored simultaneously and some of the items failed to be restored or restored with warnings. Veeam provide additional information to explain the Restore Status is displayed according to the entire restore session result.
  • New – Azure AD application used for the Restore Portal cannot be automatically configured for multiple organizations added to the Veeam Backup for Microsoft 365 scope. To make the Restore Portal work in a multitenant environment, you need to configure your Azure AD application manually.
  • New – Azure Archive Storage accounts configured for Read-Access Geo-zone-redundant Storage (RA-GZRS) replication are not supported. Veeam provide this handy Microsoft Article explaining this. If you’ve worked with Archive storage with any other Veeam solutions such as Veeam Backup for Microsoft Azure or Veeam Backup & Replication, it’s the same reason.
  • New – Veeam’s REST service can’t co-exist on a host running Veeam Backup for Microsoft 365 proxy.
  • New – Detailed in Upgrade Section Above– After upgrading to v6, backups of Exchange Online may fail with the “Unauthorized (401)” error.
  • New – Exported logs can’t be opened by the default built-in Microsoft tools if the backup job name is more than 260 characters. Workaround: Use 3rd party tool or shorter backup job names.
  • New – In the Backup Job wizard, search within sites works only by URL whilst the sites are still being enumerated.
  • New – Port 443 is not supported for creating an archiver appliance in the AWS China region.
  • New – PowerShell 7 is not supported.

 

Backup:

This list below focuses on the new issues/limitations introduced in v6, whether by the result of bugs or new feature support etc.

  • New – Backup operations can be interrupted or fail with the Exception type “System.OutOfMemoryException” error if using an SSL filtering internet proxy. Workaround: Exclude internet proxy for the Veeam components.
  • New – In Multi-Geo organizations, backup of a team with preferred data location set to a region different from the initial region may fail with the “(502) Bad Gateway” error. Workaround: None
  • New – In organizations with the renamed SharePoint domain, sites explicitly selected for backup need to be re-added to a backup job. Workaround: Re-add to backup job.

 

Backup Copy and Data Retrieval:

New to v6 is the Backup Copy functionality, Veeam have provided the following limitations/expected behaviours as well as known issues below:

  • Known/Expected Behaviour – AWS Backup Copy to Amazon S3 Glacier/Glacier Deep Archive can take 3x more time vs copying from Azure blob to Azure Archive Storage.
  • Known Limitation – Backup Copy is not supported in the China Non-regional, China East and China North Microsoft Azure regions.
  • Known/Expected Behaviour – When using Item-level retention, increasing the retention period doesn’t result in copying older restore points to Archive tier during next Backup Copy job run.
  • Known/Expected Behaviour – In certain conditions, data retrieved for either a team or its corresponding site might be available via both Veeam Explorer for Microsoft SharePoint & Veeam Explorer for Microsoft Teams. Veeam haven’t elaborated on these conditions unfortunately.
  • Known/Expected Behaviour – The number of items transferred by a copy job targeted at an archive tier repository when using Item-level retention may mismatch the number of items transferred by the corresponding backup job. Veeam haven’t elaborated if this is a visual issue or an actual data protection gap.
  • Known Issue – Under certain conditions, export of retrieved Teams posts for a selected time period may fail. Veeam haven’t elaborated on the conditions that cause this.
  • Known limitation – Availability period for retrieved data cannot be reduced if an archive tier repository with an already requested data retrieval has been removed and then re-added.

 

Reporting

A nice and simple one, there’s only one new known reporting issue and it’s purely cosmetic. All reports are created with the outdated product name “Veeam Backup for Microsoft Office 365”.

 

Restore Portal

As the restore portal is a new feature available within v6, all of the below points are new:

  • Limitation – Restore Portal is supported only for organizations added to Veeam Backup for Microsoft 365 using the modern app-only authentication method.
  • Limitation – Searching by part of a word isn’t supported. Word-prefix search is supported however.
  • Limitation – On the Restore List tab, sorting items in the list by Restore Status is not supported.
  • Limitation – Group mailbox data is not supported for explore and restore using Restore Portal.
  • Limitation – Microsoft Teams data isn’t supported for explore and restore using Restore Portal.
  • Limitation – Restore Portal allows you to restore data from the most recent restore point that is available in a backup repository.
  • Limitation – You must configure and run different restore operations for Exchange, SharePoint and OneDrive items.
  • Limitation – Backups created for on-premises Microsoft organizations cannot be restored using Restore Portal.
  • Limitation – Restore Portal allows you to restore only data from backups created for Microsoft 365 organizations when deployed for a hybrid organization.
  • Limitation – Backup Copies cannot be used for restoration from the Restore Portal.
  • Expected Behaviour – Restore sessions are not closed automatically and keep running for an hour after a user who logs into the Restore Portal refreshes their browser during the current login attempt.
  • Expected Behaviour – A user or restore operator cannot perform a restore via the Restore Portal if the corresponding organization has been removed from the Veeam Backup for Microsoft 365 scope during the Restore Portal session.

 

Veeam Explorers

The below are newly discovered issues or limitations within the v6 versions of the Veeam Explorers for Microsoft Exchange/SharePoint/Teams or OneDrive for Business:

  • Microsoft Exchange – It is not supported to restore group mailboxes or compare them to production using a username & app password for authentication.
  • KEY Microsoft SharePoint Issue – SharePoint site restore from SharePoint Online to SharePoint 2016 is not supported.
  • KEY Microsoft SharePoint Limitation – Team site restore to a Multi-Geo organization using Basic authentication is not supported.
  • KEY Microsoft SharePoint Known Behaviour– When using Modern app-only authentication, the Microsoft Graph Site.Read.All permission is required for SharePoint data restores.
  • Microsoft SharePoint – It was a known issue that SharePoint Team site restores to a new alias weren’t supported for STS#0 and STS#3 site templates, but it’s now also noted that other locations aren’t supported either for these restores.
  • Microsoft SharePoint – Restore from SharePoint Online to SharePoint Subscription edition is not supported for sites using the SITEPAGEPUBLISHING#0 and GROUP#0 templates.
  • Microsoft SharePoint – Restore of personal sites to an organization where Custom Scripts option is disabled for personal sites is not supported.
  • Microsoft SharePoint – Restore of a personal site to another location is not supported if the user account does not exist in the target organization.
  • Microsoft SharePoint – A SharePoint item with minor versions is restored with major versions if the target library or list is set for major versions only.
  • Microsoft SharePoint – SharePoint items might be incompletely restored to another location, if the item has references to other items in its original location.
  • Microsoft SharePoint – Restore of a subsite to another organization fails if this subsite uses a template that does not exist in the target organization.
  • Microsoft SharePoint – Site collection features may not be preserved upon a site restore to another organization. After restore, such features can be activated for a subsite manually in Microsoft 365.
  • Microsoft SharePoint – Approval status of SharePoint items of the Document Set type is restored to Pending if an item is restored to another location.
  • Microsoft SharePoint – Restore of Shortcuts to shared folders is not supported.
  • Microsoft SharePoint – SharePoint domain in links to a source organization are changed to a target organization after site restore to another location.
  • Microsoft SharePoint – Field value in a ‘list column’ changes to the default after restore, if the field value has not been specified.
  • Microsoft Teams – Team posts restore fails with the “App id com.microsoft.teamspace.tab.web needs to be installed to the scope … and be in an unblocked state to install/update a tab” error if the WebSite Teams App has been manually blocked via Teams Admin Center.

 

RESTful API

There’s thankfully only one FYI style notice in the release notes related to the RESTful API that action endpoints are deprecated and substituted with the corresponding new endpoints. However, Veeam have also highlighted that Version 3 of RESTful API resources have been deprecated in Veeam Backup for Microsoft 365 v6.

 

PowerShell

It’s not on the release notes but I believe it’s worth highlighting that the cmdlets haven’t changed, whilst the acronym for this product will change from VBO to VBM, the cmdlets don’t reflect this yet. For example, a GET cmdlet will still be GET-VBOxxx instead of GET-VBMxxx. At least for now, this means upgrading won’t break your scripts.

 

Conclusion

Hopefully you’ve found this write-up to be informative, this is by no means exhaustive as there’s still more to be discovered within the v6 release that I’ll find out about in the coming days. It’s also worth mentioning that my blog post regarding certificate validation errors when upgrading Veeam Backup for Office 365 v5 doesn’t impact this release, as Veeam have replaced the expired certificate, you can read more about this here.


16 comments

Userlevel 7
Badge +20

Great post Michael nice and detailed.  Mine was just to inform it was out. :yum:

Userlevel 7
Badge +12

@MicoolPaul 

Thanks for the blog. You must have used your entire afternoon to read and summarize all this information :)

Good work 👍

 

Userlevel 7
Badge +13

Wow, that's very detailed! Thanks @MicoolPaul for this great post! Agree with @Mildur, have to read it later.

Userlevel 7
Badge +12

Wow, that's very detailed! Thanks @MicoolPaul for this great post! Agree with @Mildur, have to read it later.

I have read the entire Blog yesterday. :-)

What I wanted to say was, that Michael was using the entire afternoon to write the blog 😅

Userlevel 7
Badge +20

Just to add as well, the first job run after upgrading to v6 requires additional time to perform some verifications. This doesn’t mean content is being downloaded a second time however. Subsequent runs are fine. For example, my test lab normally takes about 20-40 seconds for incremental backups. After upgrading to v6, the job run took 10 minutes, but afterwards the incremental backups are back to 20-40 seconds.

Userlevel 7
Badge +20

Just to add as well, the first job run after upgrading to v6 requires additional time to perform some verifications. This doesn’t mean content is being downloaded a second time however. Subsequent runs are fine. For example, my test lab normally takes about 20-40 seconds for incremental backups. After upgrading to v6, the job run took 10 minutes, but afterwards the incremental backups are back to 20-40 seconds.

This is good to know for sure. 👍

Userlevel 7
Badge +13

Thanks for the detailed summary @MicoolPaul I still have to deploy v6, but now I feel like I already know everything 😅

 

Userlevel 7
Badge +3

Great and most importantly very useful post, @MicoolPaul, thank you for putting this together! :nerd::clap:

Userlevel 7
Badge +20

Updated with information on the automatic update feature included in VBM365 v6!

Userlevel 7
Badge +7

Brilliant article and quite comprehensive. Thank @MicoolPaul 

Userlevel 7
Badge +13

If you’re seeing a constant CPU load after upgrading to v6, even when the server/job is in idle, there’s now a hotfix available: https://www.veeam.com/kb4294

Userlevel 7
Badge +13

Updated with information on the automatic update feature included in VBM365 v6!

Do we now say VBM365 instead of VBO365? It will get confusing, depending on which version you’re running 😉

Userlevel 7
Badge +20

Updated with information on the automatic update feature included in VBM365 v6!

Do we now say VBM365 instead of VBO365? It will get confusing, depending on which version you’re running 😉

I am pretty sure it is now VBM since Veeam rebranded it.

Userlevel 7
Badge +20

Yep I’m saying VBM365 now, as that’ll be the way it is until the next inevitable name change from Microsoft. Maybe Microsoft 356 when they realise that’s nearer their actual uptime. 🙄

There’s been a few hotfix/KB articles out from Veeam recently such as issues upgrading when leveraging REST etc. I’m going to collate these into another post/comment.

Userlevel 7
Badge +20

Yep I’m saying VBM365 now, as that’ll be the way it is until the next inevitable name change from Microsoft. Maybe Microsoft 356 when they realise that’s nearer their actual uptime. 🙄

There’s been a few hotfix/KB articles out from Veeam recently such as issues upgrading when leveraging REST etc. I’m going to collate these into another post/comment.

Look forward to this as the one about high CPU usage is concerning.

Userlevel 1

In v5 it is not possible to restore sharepoint data/site into another tenant. After upgrading to v6, can we restore the old data to another tenant or should the backup created with v6?

 

Comment