Offsite backup as a Ransomware protection strategy


Userlevel 7
Badge +7

There has been a significant focus on how to make Veeam Backup resilient to ransomware.  More specifically - how to have Veeam backups available for recovery in case of an incident.

In the past few months, I have had discussions with customers who have successfully recovered their production data after a ransomware attack. The common theme among these successful recoveries has been the availability of a secondary off-site copy of their backups to recover from, as their primary backup repository was compromised and destroyed outside of the Veeam software.

Veeam ransomware protection should be the customers’ focus, and it is always a priority goal in my architecture designs. Secure by design is also a core design principle of the Veeam Architecture team.

Off-site backup copies can provide an additional layer of data security:

  • They are usually behind a VPN with a robust firewall deployment
  • They are typically kept on a different network segment
  • They often use an additional authentication method (different AD domain, IAM auth, etc.).

Veeam has many ways to create a secondary off-site backup.

Backup copy job (BCJ):

  • A backup copy job makes a copy of a Veeam Backup job file to a different repository at a secondary off-site location. 
  • A backup copy can be kept locally, but that deviates from the purpose of having an offsite replica to restore from. 
  • The backup copy can have a different retention period than the backup file and be kept on other media.  In Veeam Backup and Replication V12, a backup copy can be sent to object storage - either on-premises or in the cloud.
  • It can be run immediately after a backup job finishes
  • The backup repository can be a hardened repository with immutability.

Backup to tape:

  •  A backup copy job to tape is like a backup copy job, as it copies the backup files to a different media set
  •  A backup-to-tape job can have a different retention period than the primary backup job, and tapes can be kept outside a library longer than a set retention policy.
  • It can be run immediately after a backup job finishes
  • The target is a tape device - a tape library or a standard lone tape device.  
  • The tape can be an immutable write once read many (WORM) device
  • After a backup copy job to tape is completed, the tape(s) must be removed from the library to provide a total air-gapped copy of the data.

Scale-Out Backup Repository (SOBR) with Capacity Tier:

  • Capacity Tier is an object storage extent.  This object storage can be on-premises or in the cloud
  • Backup data is automatically off-loaded (moved and copied) to the object storage
  • Backup copies can be made immutable with an S3 object lock.  Many S3-compatible object storage providers provide this option.

Veeam Cloud Connect (VCC):

  • The backup copies are sent off-site to a Veeam Cloud provider to be stored in their secure infrastructure.
  • There is a large Veeam ecosystem of providers across the globe.
  • The backup VCC repository can be a hardened repository with immutability.
  • It can be run immediately after a backup job finishes
  • VCC can also be hosted in a large enterprise that provides private cloud services to its departments and divisions without needing an external provider.

Replication

  • Veeam Backup and replication provides  a VM snapshot-based replication process and continuous data protection (CDP) replication. 
  • Both replication jobs after a swift RTO.
  • CDP offers an RPO of seconds.
  • Veeam Replication is great for quickly powering up VMs at a secondary location during a system outage.
  • Replication is not the best choice for recovering from a ransomware event, as any change to a source VM will automatically get replicated to the secondary site.

Conclusion
The second copy of the backup jobs being kept off-site or in a secure secondary environment separated from production is a core development tenet of Veeam Software. It will continue to be enhanced and expanded in all future Veeam Software releases, regardless of the future's repository types.


Every new Veeam Backup deployment or re-architecture should have a backup copy included.
 

 


17 comments

Userlevel 7
Badge +20

Really great post Joe. 👍

Userlevel 7
Badge +12

After a backup copy job to tape is completed, the tape(s) must be removed from the library to provide a total air-gapped copy of the data.

This is really important for tape backups, but most often overlooked. Remove your tapes or someone will erase them…

Great and informative post @vmJoe!

Userlevel 7
Badge +11

Backup copy is the law… 👮🏼

Excellent post Joe.

Userlevel 7
Badge +7

@regnor I am always amazed at the customers who leave a weeks worth of newly written tapes in there library - 😱.  They will be the ones who will complain when they loose data during a disaster.

Userlevel 7
Badge +7

Backup copy is the law… 👮🏼!

@wolff.mateus How true and one that quite a few people don’t think about.

 

Userlevel 7
Badge +12

@regnor I am always amazed at the customers who leave a weeks worth of newly written tapes in there library - 😱.  They will be the ones who will complain when they loose data during a disaster.

Absolutely. I often see weekly or monthly rotations; and when asking about how much data they're willing to lose, you hear a day or less...

Userlevel 5
Badge +4

Well done, Joe. Clear, simple, easy to achieve.

Userlevel 7
Badge +8

We need an Immutable Tape solution so we can leave tapes in the library. Perhaps a setting that can only be configured on the physical console of the library to set over write policies. haha

 

Weekly exports become a challenge for myself, but I should get back on that. 

Userlevel 7
Badge +7

We need an Immutable Tape solution so we can leave tapes in the library. Perhaps a setting that can only be configured on the physical console of the library to set over write policies. haha

 

Weekly exports become a challenge for myself, but I should get back on that. 

Well - you could use WORM media if you want to leave tapes in a library.  But thruthfully why?  I realize that swapping tapes is time consuming - I did ny myself for 10 years.  But the benefit of having a true copy of your data not connected to a network or visible to the world can provide huge dividends in a disaster.

Userlevel 7
Badge +17

Worm media is a waste of resources…

You have these two options - less comfort and more security or more comfort and less security.
The immutable object storage is an option, but a completely offsite tape copy in another location is even more secure. Then hackers can even capture all of your systems, they will not get at your backup copy… But it comes with effort and much less comfort.

Userlevel 7
Badge +12

Does anyone still use WORM tapes? I haven't seen or heard from them in years.

Userlevel 7
Badge +17

Does anyone still use WORM tapes? I haven't seen or heard from them in years.

NO! 😎

Userlevel 7
Badge +8

We need an Immutable Tape solution so we can leave tapes in the library. Perhaps a setting that can only be configured on the physical console of the library to set over write policies. haha

 

Weekly exports become a challenge for myself, but I should get back on that. 

Do you know Active Vault or Ransomware block from Quantum?

Userlevel 7
Badge +7

We need an Immutable Tape solution so we can leave tapes in the library. Perhaps a setting that can only be configured on the physical console of the library to set over write policies. haha

 

Weekly exports become a challenge for myself, but I should get back on that. 

Do you know Active Vault or Ransomware block from Quantum?

If you leave a tape in a library it can be found in inventory, loaded to a tape and accessed - read, or data can be deleted from it. The purpose of tape is to have a transportable media that you can take out of a library and store somewhere.  If it’s off the network (physically outside a library) who can access the stored data?  If it’s in the library or accessible over a network someone will be able to get to the data.
 

if you don’t want to export the tape why not just send data to object storage in a different location (S3, Azure, Wasabi, GCP, etc) at least it is behind a separate firewall and uses a different authentication method and Veeam can provide some immutability.

 

 

Userlevel 7
Badge +8

@vmJoe Ok on Active Vault partition, Veeam can’t access to it but i agree can be available through network (web interface). This is not the case for ransom block.

Userlevel 7
Badge +22

Excellent post!. I have heard people using the term air-gapped backup for Tape but be careful, the official NIST definitions states there cannot be any automated transferring of data, only under human control:

https://csrc.nist.gov/glossary/term/air_gap#:~:text=Definition(s)%3A,manually%2C%20under%20human%20control).

“An interface between two systems at which (a) they are not connected physically and (b) any logical connection is not automated (i.e., data is transferred through the interface only manually, under human control).”

Userlevel 7
Badge +16

Another good post @vmJoe . Personally, I use the last option you list (Replication), in addition to Storage Replication, and even another layer - offsite Immutable Backup. Great solution options, but none of them matter if the ransomware is withIN the image backups themselves...altho, hopefully a very small potential for that to be.

Cheers!

Comment