The overall intent of this article is to consolidate major considerations when it comes to using Veeam Recovery Orchestrator to Recover from Backup Copies.
Key:
- VBR: Veeam Backup & Replication
- VRO: Veeam Recovery Orchestrator
Orchestrating recovery from Backup Copies - Use-Case Overview:
- Below you will see a is common customer design example.
- Both sites are Active/Active with production virtual workloads being protected with Backup Jobs as well as Backup Copy Jobs. (BCJ)
- (1) VBR server at Site-Z which is the DR location.
- Backup Proxy at each location.
-
(1) Backup Repository for Primary Backups at each location.
-
(1) Backup Repository for Backup Copies at each location.
- (1) vCenter at Site-Z and vSpheres cluster at each location.
- Agent deployed protecting a physical workload.
- Our Veeam Transport (Data Mover) service is represented by a (M) icon here for these machines.
- Our Agent services are represented by a (A), although the Agent Backup and the one on VBR server is an Orchestrator Agent, they are different services.
- Backup Proxies:
- Virtual or Physical Proxy server transporting/source data from the Production vSphere clusters at each location.
- Backup Repositories:
- These are deployed at each location, generally a Windows/Linux server with direct attached storage, NFS/SMB or Storage appliance specific.
- RPO-1-A is the backup repository for Site-A VMs and Agent Backups.
-
RPO-1-Z is the backup repository for Site-Z VMs.
-
RPO-2-A is the backup repository sitting at Site-Z for copies of backups.
-
RPO-2-Z is the backup repository sitting at Site-A for copies of backups.
- This is also often the Mount server, which is used to mount data for instant recovery with our vNFS protocol.
- You will also see Gateway mentioned here, this is specific for a certain backup repository which are unable to have the Data Mover installed locally. (IE: NFS/SMB)
- Note: This representation is acting a Direct Attached Storage on a Virtual Machine which has the Data Mover installed locally.
- Enterprise Manager and Veeam Recovery Orchestrator:
- For this example, we have these management overlay servers in the cloud, it's best when they are not on-prem if a site fails.
- Even better would be to have VBR (VBR-Z) also in the cloud, but for the sake of common configurations we will leave it at Site-Z.
- There are also considerations regarding Virtual Labs and testing that will come up in later articles I plan to develop.
Additional Site Recovery Considerations:
-
Backup Copies are for recovering data from one site to another during a critical site failure, we can orchestrate recovery of VMs from Site-A to Site-Z.
-
If Site-Z fails, VBR-Z is sitting on location, Veeam Recovery Orchestrator requires VBR to be online for recovery, as we need access to the policies and repositories.
-
There are ways around this, as previously mentioned having your VBR server in the cloud or witness location.
-
Replicating the VBR server with Veeam and doing a manual recovery of the VM at the other location.
-
Taking a configuration backup of Veeam, having a cold server in-place at Site-A, cloud or witness site to recover the VBR server during an orchestration event.
-
Note: That this failure will cause an increased RTO for a Site-Z failure, Site-A will not be impacted during a critical site failure as VBR will be online.
-
-
-
We often use Source and Target to represent taking a workload from Site-A (source) and recovering it to Site-Z (Target) and vice versa.
How do Restores from Backup Copies work: (Referencing the above image)
Sourcing Data:
- VBR-Z has backup copy policies configured to protect both Site-A and Site-Z.
-
When the backup copying process starts, Veeam Backup & Replication accesses backup files in the source backup repository, retrieves data blocks for a specific machine from the backup file, copies them to the target backup repository, and composes copied blocks into a backup file in the target backup repository.
-
The backup copying process does not affect virtual and physical infrastructure resources, does not require creation of additional VM snapshots or VSS snapshots and does not produce load on machines whose backups are copied.
-
Agent Backup has its own in-guest OS Agent installed that facilitates source and moving data to the onsite repository.
-
But we can still create a Backup Copy operation just like we would for a Virtual Backup.
-
-
The backup copy chains/data are stored on RPO-2-A and RPO-2-Z respectively.
-
Create a Backup Copy Job: https://helpcenter.veeam.com/docs/backup/vsphere/backup_copy_create.html?ver=120
Restoring Data:
-
We have multiple ways to restore data in Veeam, the one we will focus on is VM Restore.
-
VM Restore rehydrates the entire VM from the backup Repository with our Transport service into a ESXi Host and Datastore you specify during the recovery operation.
-
Veeam Backup Repository location: (Restore Source)
- One of the most important design aspects is the location of the backup data in which you plan to restore from.
-
The best option here is a backup copy repository is to be as close to the Target vSphere environment as possible for recovering those workloads back to the hypervisor.
- Note: Pulling from a Data Domain appliance which needs to rehydrate globally deduplicated data could add restore latency.
-
Veeam Recovery Orchestrator:
- It's important to note that for Veeam Recovery Orchestrator (VRO) we are utilizing components created and configured from the Veeam Backup & Replication interface.
- Veeam Recovery Orchestrator is an additional product and deployment on top of your existing Veeam Backup & Replication server, it connects to your Enterprise Manager and/or VBR instances and creates orchestration overlays.
- VRO Placement: If you have presence in the cloud, we would recommend that this be placed in the cloud or at a Witness location.
- Additional considerations: Veeam University (VU) https://veeam.looop.co/topic/834185
- VMware Tagging: A popular but very important aspect of our Veeam product when grouping and categorizing workloads to protect and in this case grouping of servers for restoration.
- With Veeam Recovery Orchestrator mapping and grouping your workloads for orchestration, it's best practice to-do so with VMware Tags.
Veeam Recovery Orchestrator Core Components:
- Recovery Locations: We create and utilize these in VRO to pinpoint the resources we are going to utilizing during a recovery event. Some of the selections include source data and placement of converted workloads. (Compute, Storage, Networking and Re-IP Rules.)
- Recovery Plans: A recovery plan is where we select the workloads we will orchestrate and the Recovery Location in which those workloads will be recovered to.
How to Configure VRO for Orchestration of a Test VM from Site-A to Site-Z:
-
Below I am going to walk you through how to configure an example VM recovery of a Site-A VM being recovered to Site-Z.
-
We will configure VMware Tags, Recovery Locations, and Recovery Plans.
-
Note: Agent Recovery, you can't use VMware Tags but you can select Agent Backup policies protecting that VM instead.
-
Creating VMware Tags:
- We need to create at a minimum (3) VMware tags for each Site on our vCenters or single vCenter if it manages both sites.
- These Tags will be used to define Compute and Storage locations for the entire VM restore of a Virtual or Agent workloads.
- Create Tags:
- In vCenter you will want to go to your menu in the upper left-hand corner and then select (Tags & Custom Attributes) from the list.
- Create a new Category and name it VRO or something similar.
-
Create (3) New Tags for each location and associate them to the VRO Category you created. (IE: VRO_COMPUTE_SITE-Z, VRO_STORAGE_SITE-Z, and VRO_TESTVM_SITE-A)
-
Note: For this configuration test to work we need Tags for Compute and Storage at the Target (Site-Z) and a Tag at the Source (Site-A) site identifying the test VM.
- Reference: I would link a VMware article but that might change, so google "What are VMware Tags and How to use them"
-
- Associate Tags:
- Go back to Inventory on the upper left-hand menu in vCenter and find your Compute and Storage to Tag.
- Associate the compute Tag you created to an ESXi host or Cluster.
- Associate the Storage Tag you created to a Datastore or Cluster.
- Associate the TESTVM Tag to a Test server that you have protected with a Backup Job and Backup Copy Job on the VBR server.
- Updating VRO Inventory:
- Use the Tags created above, if you don't see them in the VRO inventory, login to the Veeam One Web client (https://VROVM:1239/) and run a rescan from the configuration page.
Creating Recovery Locations:
- Now that we've identified the Compute and Storage locations, we can now use these VMware Tags in our Recovery Location in Veeam Recovery Orchestrator.
- Login to your VRO server: (https://VROVM:9898), click the gear icon in the upper right-hand corner to enter Config/Administration mode.
- On the left-hand side in the Navigation pane click (Recovery Locations), then click (Add).
- Once in the New Recovery Location Wizard fill out the following:
- Select (VMware) as the Location Type.
-
Give a Location Name: (IE: VRO_RECOVERY_LOCATION_SITE-Z)
- Recovery Options: Select (Recovery Agent Backups) and/or (Recover VM Backups) as needed.
-
Compute Resources: Select the (VRO_COMPUTE_SITE-Z) Tag we created earlier.
-
Storage Resource: Select the (VRO_STORAGE_SITE-Z) Tag we created earlier.
-
Storage Options: Make sure to SELECT backup copies, this is for cross site recovery. (%80 of Capacity is fine)
-
Agent / VM Networks: This will map source VMware Network/Port Groups, to the Target VMware Network/Port Groups as needed.
- Note: We capture the Port Group the VM is on during the backup which is stored in the .VMX file.
-
Re-IP Rules: This is only required if you are restoring to a different destination Port Group/network.
-
Note: If you don't have a stretched Layer 2 you may want to consider this.
-
Note: Often, we have customers who replicate their BGP routes and VLAN configurations between sites and allow for a L3 cutover instead.
-
- Data Sovereignty: This is defined in VBR and can be enforced here in VRO If needed.
Creating Recovery Plans:
- Now that we've created a Recovery Location, we will create a Recovery Plan that will associate to our newly created Recovery Location.
- Exit Administrator by clicking (Exit Administration) in the upper lefthand corner.
- On the left-hand side in the Navigation pane click (Recovery Plans), select (Manage), and select (New)
- Once in the New Restore Plan Wizard fill out the following:
-
Give a Recovery Plan Name: (IE: VRO_RECOVERY_PLAN_SITE-AtoZ)
- Plan Type: Select (Restore) plan type.
- Recovery Location: Select the (IE: VRO_RECOVERY_LOCATION_SITE-Z) Recovery Location that we created.
- Inventory Groups: This is where we can select the (VRO_TESTVM_SITE-A) Tag we created.
-
Note: You can also select the Backup Copy Job we created as well, tags are best for scaled out deployments or single VM testing.
-
- VM Recovery Options: Choose defaults here.
- VM Steps: Add Ping Test
- Protect Inventory Groups: Only for reprotection of the Virtual Machine after a commit failover.
- RTO&RPO: This simply validates whether the protected workloads are achieving these marks for the associated Backup Copy Plan.
- Report Template: Default
- Report Scheduling: Default
Closing:
- Once you have this all created you are ready to run a live recovery of a Virtual Machine.
- This machine you backed up and then created an offsite copy with a Backup Copy Job with Veeam Backup & Replication.
- Note: This is a live recovery and should be taken at your own risk, Veeam is not responsible for any production outages or loss of data for your recovery configurations and actions.
Recovery_Workbook: https://docs.google.com/spreadsheets/d/1qyZHWLFMg64q9CfQ3Ohnv1gRWNzT9LSzpW7L8JKkCEQ/edit?usp=sharing
