Veeam Recovery Orchestrator and Linux Re-IP

Introduction

While working with Veeam customers recently who wanted to automate Linux re-IP as part of their recovery process using Veeam Recovery Orchestrator (VRO), I started into some exploration to see if I could leverage any existing scripts that could be easily modified to accomplish this. I put my Google Fu to work and found numerous Linux re-IP scripts available online. Several such scripts have been specifically designed to work with Veeam during recovery or failover scenarios. However, when I started building out my lab and testing with some of these, I discovered that the existing scripts either rely on methods specific to certain Linux distributions or employ outdated/deprecated techniques to accomplish the re-IP.

Given the diversity of Linux distributions and network management tools, the exact method for re-IP can vary depending on the specific environment. My goal was to find a common method that would work for the foreseeable future and across multiple Linux distributions. I decided to start into creating and testing a new script using Network Manager as the method of choice.

Network Manager is installed by default in newer versions of RHEL (7+) and its branches (Rocky, Alma, CentOS, etc.) and in the latest releases of Ubuntu. Network Manager can also be installed and configured in other distributions as well. More info on Network Manager can be found here: https://networkmanager.dev/

To make these changes inside of a VM, we need a way to inject the Network Manager (nmcli) scripts into the VM Guest OS and either persist through or reconnect after the changes are made to verify the outcome. Since VRO currently only supports vSphere VM workloads, the script utilizes VMware PowerCLI to perform the in-guest scripting.  

The power of VRO in this scenario is that we can dynamically pass the VM info and credentials into the script and run multiple re-IP plan steps in parallel on a per-VM basis to make this scale up to fit our recovery plan needs and improve overall RTO.

The script and basic instructions can be found here: https://github.com/VeeamHub/veeam-orchestrator/tree/main/ReIPLinux/Original

For a detailed look into how this is configured and used, keep reading 😁.

Prerequisites

• VM Guest OS
  • Network Manager needs to be installed and running
    • The installation steps for your specific Linux distribution should be searched for online if it is not pre-installed
  • Network Manager needs to be managing the network devices
    • Use NMCLI to verify managed devices and their status: https://networkmanager.dev/docs/api/latest/nmcli.html
• Veeam Backup and Replication server or Veeam Recovery Orchestrator server (depending on script location setting)
  • PowerCLI must be installed
    • https://docs.vmware.com/en/VMware-PowerCLI/latest/powercli/GUID-2DD2454B-2B1E-4A7D-9134-B442254F0681.html
• All necessary credentials will need to be managed in VRO
  • https://helpcenter.veeam.com/docs/vro/userguide/adding_credentials_manually.html

Configuration

• Verify that the prerequisites are met
• Download the PS1 script from Veeamhub and place it on the VRO server
  • https://github.com/VeeamHub/veeam-orchestrator/tree/main/ReIPLinux
• Create a new Plan Step - https://helpcenter.veeam.com/docs/vro/userguide/uploading_scripts.html
  • Give the new Plan Step a name

    

     

  • Browse to the script path to import

    

     

  • Add the following required parameters

    Name	Description	Type	Default Value
    vCenterServer	Hostname, FQDN, or IP of the vCenter Server that manages the target VM. This is used for PowerCLI connection.	Text	(Set a default value as needed)
    vCenterServerCreds	The credential used to authenticate to the "vCenterServer"	Credential	(Set a default value as needed)
    GuestOsCreds	The credential used to authenticate in the VM guest OS	Credential	(Set a default value as needed)
    VMName	Name of the target VM	Text	%current_vm_name%
    VMOrigIP	Original IP Address. Used to locate the network device to be modified.	Text	%current_vm_ip%
    ReIPRule	Re-IP Rule to apply. Can use asterisk(*) to keep source IP values for an octet. Ex. 10.1.*.* or 10.0.1.*	Text	(Set a default value as needed)
    CIDR	CIDR number for subnet mask. Ex. 24, 28, etc.	Text	(Set a default value as needed)
    NewGateway	New gateway address.	Text	(Set a default value as needed)
    PrimaryDNS	Primary DNS address.	Text	(Set a default value as needed)
    SecondaryDNS	Secondary DNS address.	Text	(Set a default value as needed. You can leave this blank if no SecondaryDNS is needed)

     

  • (Optional) The following parameters are not required but can be used if necessary

    Name	Description	Type	Default Value
    SudoRequired	Use this parameter to force the use of Sudo.	Text	true (the script defaults to false unless this parameter is set to 'true')
    SudoPassRequired	Use this parameter to specify if a password is not required. The script defaults to using a password with Sudo if 'SudoRequired' is set to true.	Text	false (the script defaults to true unless this parameter is set to 'false')

     

  • For most errors that may occur, a retry will not resolve the issue. So to save time on the plan completion, I have set the “Retries” to 1. After all of the parameters are set it should look like the following:

    

     

  • The scope can be defined as needed on the “Impact” tab. I left this in the Default Scope for my use in testing.

  • Review the settings and then “Finish” to save this new Plan Step

• Now that this Plan Step exists, we can add it to a Recovery Plan. There are a couple of ways to do this.
  • The new re-IP step can be added to “VM Steps” when creating a new plan or when editing the Properties of an Inventory Group within an existing plan.
    • Creating Replica Plans - https://helpcenter.veeam.com/docs/vro/userguide/replica_plan_steps.html
    • Creating Recovery Plans - https://helpcenter.veeam.com/docs/vro/userguide/restore_plan_steps.html
  • You can also add steps per-VM within the Recovery Plan
    • Configuring Steps - https://helpcenter.veeam.com/docs/vro/userguide/configuring_steps.html
• Set the parameter values needed per-VM or for all VMs in a plan (multi-select is available)
  • Configuring Step Parameters - https://helpcenter.veeam.com/docs/vro/userguide/configuring_step_parameters.html
• Run the plan or a DataLabs test to verify recoverability.
  • If running a DataLabs test, you will need to set the following value to “Execute” either in the Plan Step itself or in the Recovery Plan.

    

     

  • This can be scheduled to run automatically
    • Configuring Test Scheduling - https://helpcenter.veeam.com/docs/vro/userguide/configuring_test_scheduling.html
  • Verify the results. You can review the results in the console or by generating the Full Report version of the Plan’s Execution Report or the DataLab Test Results for per-VM details of the script execution.
    • Generating Reports - https://helpcenter.veeam.com/docs/vro/userguide/generating_reports.html

Conclusion

By leveraging PowerCLI and Network Manager (nmcli), we can automate the re-IPing of VMware Linux VMs during recovery and failover operations. This automation streamlines the process, reduces recovery time objectives, and enables efficient testing and verification of recovery plans. Detailed reports can be generated to provide insights into the effectiveness of our recovery strategies.