All-Demo Session & Installable ISO for Veeam Hardened Repository

All-Demo Session & Installable ISO for Veeam Hardened Repository
Userlevel 7
Badge +10

 

22-February 2024 Update:  GOOD NEWS! We are scheming a big update and overhaul of this page at or around VeeamON time. I can’t say more!

 

During the lead-up to VeeamON, it was amazing to see what content was leading the pre-registration information. We have the most data here than ever before. Constantly back and forth with the “Architecting Veeam Backup for Microsoft 365 at Scale” session, the session that @hannesk, Christoph Meyer and I are delivering was racing to the finish line as the leader. The “All Demo Session for Veeam Hardened Repository” was in the end the most popular pre-registered session, and as promised in the delivery in Miami, here is your everything you need to know guide.

Post Publish Updates:

  • 1-June 2023:  The installation requirements for the .ISO have been added.
  • 1-June 2023:  Video of installation of the .ISO has been added.
  • 2-June 2023:  Clarification on support, formatting of document.
  • 5-June 2023:  Updated .ISO with support banner and logon screen update, link updated. No changes to Ubuntu configuration or hardening script.
  • 29-June 2023: Updated .ISO with one enhancement, offline installation works.

Recording Resources

You can watch one of my practice runs of the session from Miami here:

At the VeeamON virtual event experience, we will have the replay from Miami session up very soon. 

Markdown File & Write Up

I have made a markdown file with a lot of the key information ready, you can download it here: rickvanover/VeeamHardenedRepoHub (github.com) Additionally - you can download the Markdown file at the QR code below:
 

This link goes to the VHR Markdown file that I’ve written up.

If you are not doing Markdown or GitHub you can download this as a PDF at the bottom of the page.

Overall advice

My overall advice on the Veeam Hardened Repository is to plan your deployment, this will greatly improve your experience with using the VHR. For example, I’m notorious for not documenting the options I put during the Ubuntu installation at this important stage:

 

Installable .ISO & Hardening Script

Key Update 18-December 2023: We have taken down the link for the .ISO, there will be a  new one coming “Soon” :)! 

One of the key parts of the Miami experience is the update we had from Product Management on new functionality for the Veeam hardened repository, which includes the Hardening Script and the installable .ISO. I recommend watching the replay to hear from Hannes and Christop directly before using the script and .ISO. We recommend at least 100 GB for storage. You can get both below:

  • Hardening Script: https://www.veeam.com/sys507
  • Installable .ISO: )18-December 2023 - We’ve removed this download and a new one is coming “soon”!)

About this installable .ISO…. The principal benefit of the installable .ISO is that it sets up much of the installation pre-defined *and* applies the above linked hardening script. 

Installable .ISO Requirements

For the installable .ISO, we recommend you follow these prerequisites (mainly relevant if you try to install it in a VM for lab purposes):

  • This is now optional from the 29-June update:  Internet connection (HTTP to the Ubuntu update servers for automatic security updates)
  • 2 CPUs, 6 GB RAM
  • UEFI boot is required (BIOS does not work) UEFI secure boot strongly recommended
  • 1 disk with at least 100GB for the operating system (plus cache files)
  • 100GB is a hard requirement. If that’s too high, please give feedback below.
  • Then there should be at least one second disk that is larger 100GB for backup data that would be the Veeam repository

During the installation, follow the wizard and reboot after the installation finished. Then install the hardened repository role. The VHR final layout will have all disks are mounted to /mnt/backup<n>

Important: Reboot again (this second reboot locks down the whole system and you can only reboot / shutdown with sudo. there is no root / sudo anymore at this stage) - this is the result of the hardening.  Think of this result as taking a massive green pill that tells this system its role in life forever on is to be a VHR.

After reading all of this, you are now ready to watch my video of installing the VHR with the .ISO:



Installable .ISO General Information & Support Statement

  • It's a community project. please don't call Veeam support for use of the .ISO. Once a VHR is deployed and configured in Veeam Backup & Replication, it would be subject to support. 
  • Automatic updates are enabled.
  • The user does not have permissions to update manually.
  • We plan to add a note to the banner message where to find more information about the ISO installer.

Get Started with the Veeam Hardened Repository 

Getting started with the VHR doesn’t have to be overwhelming. Give it a try - and let us know how it goes in the comments below.


This topic has been closed for comments

199 comments

Userlevel 7
Badge +17

I used a VM & was successful in installing it….I just had to remember to change the BIOS section to EFI in the VM Options tab.

Userlevel 7
Badge +14

Then I would check the blog post from above. Or an easier solution, boot a live CD and clean up the files from there 😉

Running the VHR inside VMs isn't recommended as an attacker could just delete the VM including it's storage.

Userlevel 7
Badge +10

Great session. I followed the link but I didn’t see the customized Linux iso install used at the end of the demo. Thanks for sharing.

Said

Senior Network Admin

I fixed the link! Thanks.

Userlevel 7
Badge +14

The pre-configured hardened ISO will be a gamechanger. Although the manual setup hasn't been so complicated, Microsoft only shops with limited Linux skills will now likelier deploy the VHR. And so we will see a higher adoption rate and less successful ransomware attacks. 👏

Userlevel 7
Badge +22

Very interesting. As someone said this could lead to mass usage. The only danger I see here is that it could get “windows only” admins to install this without them making an effort to learn at least some linux. I have run into situations with customers when say the Linux person left the company and the poor people left behind are like stranded on a desert island. The dreaded words that I have heard a few times on the service provider side of things is “would you mind taking a quick peak at this server, something does not look right and it has no GUI” :( 

Userlevel 5
Badge +2

thanks for confirming that the problem can be solved with internet connection. I got a different error.

yes, internet is a hard requirement. otherwise the system would not get security updates. that’s why the wizard for the HTTP proxy exists

as we got so much feedback now about “no internet connection” problems: we are working to include all required packages directly into the ISO.

That creates a new problem with missing security updates. I guess, it’s something to ignore from our side, because it’s obvious that one cannot have updates without internet (assuming that nobody who uses the ISO would run it’s own Ubuntu mirror)

Has anyone found a solution to the Dell R-series servers erroring out as soon as the network connections show?  I have an R720 that will install fine with the vanilla 20.04 LTS, but errors out with this .ISO.  I have tried several different suggestions on this thread, but nothing seems to be working.  I’ve even tried changing from the Intel NIC I was using to a spare Broadcom unit I had laying around, but it didn’t seem to help.

Server specs:

  • Dell R720
  • PERC H710P RAID controller
  • Intel X520 NIC
  • Running in UEFI mode
  • 112 GB SSD Boot Mirror
  • 73 TB Storage Array

If the .ISO won’t work, what do I need to do before I use the hardening script?

Thanks!

Bob

Userlevel 5
Badge +2

Hello Bob,
unfortunately I have no idea how to solve it with the ISO. But I put your configuration to the bugs list.

https://www.veeam.com/blog/backup-repository-security-disa-stig-ubuntu-step-by-step-guide.html describes how to use the script manually.

Best regards,
Hannes

Userlevel 5
Badge +2

yes, just create a new backup server and re-connect to the Hardened Repository. 

 

Userlevel 7
Badge +14

Oh sorry. I was talking about a manual Linux installation in parallel to the Windows server. The VHR ISO itself won't work in dual boot as it completely wipes the server including the existing windows installation.

Userlevel 7
Badge +14

This should be possible. You install the Windows Server first, and afterwards do the Ubuntu setup which should detect the existing Windows installation and include it in the boot manager. Then you can do the hardening and setup the hardened repository.

Userlevel 7
Badge +10

It seems like plenty of resources @Chris.Childerhose  → What does the plain Ubuntul 20.04 install media do on the same equipment?

The plain one works fine.  I am wondering if it has anything to do with having to use a VLAN?  When I do this, it creates another entry - so does the IP then go on the main NIC or the new VLAN one that is associated to the primary?

It seems it is a VM, but you are passing the VLAN assignment thru to the install?

Userlevel 7
Badge +20

This was an amazing session. Great things coming from Veeam.

Userlevel 7
Badge +10

FYI I am closing this thread as we are close with V12.1 - and after 12.1 goes GA - the .ISO will be back on deck for new release and we’ll make a new post for that build’s feedback.

Great session. I followed the link but I didn’t see the customized Linux iso install used at the end of the demo. Thanks for sharing.

Said

Senior Network Admin

Userlevel 7
Badge +17

Hi @Assalihin - do you have (Legacy) BIOS enabled by chance? The install won’t work except for EFI-enabled machines, beit physical or VM.

Userlevel 5
Badge +2

thanks, we will check the proxy behavior. I’m a bit concerned about the IPv6 messages though…

 

Anyway: I got a a new ISO that does not require internet and I want to test it next week before Rick can update the download link

Userlevel 5
Badge +2

the new feature is: it works offline. No other changes :-)

Userlevel 7
Badge +14

@Chris.Childerhose Ubuntu 20.04 LTS is supported till May 2025. So from a security perspective it's not necessary to upgrade.

Userlevel 2

It is very good tools, thank you very much.

I tried to install on my ESxi and I cant make a connection between the Veeam backup and VHR as you can see below, what I missed her? Thank you

 

 

Userlevel 7
Badge +10

It is very good tools, thank you very much.

I tried to install on my ESxi and I cant make a connection between the Veeam backup and VHR as you can see below, what I missed her? Thank you

 

 

Hi George, did you reboot it after the .ISO the second time? Then the hardening is applied and SSH is gone.

Userlevel 7
Badge +10

The installer errors out as soon as I reach the network config screen early in the setup process.

Trying to install it on a Dell PowerEdge R720 we had lying around.  Downloaded the latest Ubuntu Server 22 LTS and it works fine, so not sure what is wrong with this install image for this particular hardware?

Hey @Graham → can you PM me the Service Tag # since it is a Dell and we’ll look up the inventory and check on our side?

Userlevel 5
Badge +2

The installer errors out as soon as I reach the network config screen early in the setup process.

Trying to install it on a Dell PowerEdge R720 we had lying around.  Downloaded the latest Ubuntu Server 22 LTS and it works fine, so not sure what is wrong with this install image for this particular hardware?

@Graham : can you maybe test a Ubuntu 20.04 installer whether it detects the network card? I remember the R720 from one of my earlier jobs and they normally work fine with Ubuntu. 

Upgrading the ISO to Ubuntu 22.04 depends on Canonical / DISA providing STIGs. In best case, I guess that’s end of the year.

Userlevel 7
Badge +10

Hi @Chris.Palmisano.26 

RE: Automatic Updates: Then they will fail/run without the ability to check.

 

RE: update manually:  This isn’t panned at this time as far as I know.

 

Will share with everyone a new .ISO is coming soon with some additional enhancements.

Hi @Chris.Palmisano.26 

RE: Automatic Updates: Then they will fail/run without the ability to check.

​​​​​RE: update manually:  This isn’t panned at this time as far as I know.


​​So just to be clear then without internet connectivity we won’t be able to patch these systems?


Will share with everyone a new .ISO is coming soon with some additional enhancements.

​​​​​Look forward to the new enhancements

@Rick Vanover  thanks for the information and quick response!