I speak to people often about ransomware and Veeam ONE has a helping hand here that can do more than many people think. What is important however is that the alarms and reports that matter get to the right persona are made visible. In this post I want to cover the 4 alarms and 1 report that matter the most AND how to handle the conditions that make these happen. Let’s get into it, these simple steps will make a different in getting Veeam ONE’s ransomware or malicious activity detected quickly.
Possible Ransomware Activity Alarm
This alarm is in the Alarm Management section for both VMware and Hyper-V and looks at two sets of rules: sustained high CPU and disk writes; but also looks at sustained network transfer (exfiltration). At a minimum, this alarm should have an email sent to two or more persons (backup admins, security team, etc.) via email and separate the emails with a space:
I also recommend going into the actions section, and put in a script that does something relevant to your team’s cybersecurity practice. For example if you have a SIEM platform; create a small PowerShell script that will alert that platform of a possible ransomware activity to draw attention there to the matter. A view of this is below:
Here you can find all the information that Veeam ONE creates on every single alarm, and the attribute which you can use as part of your PowerShell Script to build an alarm that is legible, and according to your expectations :
- %1 – Alarm name
- %2 – Affected object name
- %3 – Alarm summary
- %4 – Time
- %5 – Alarm status
- %6 – Previous alarm status
- %7 – Alarm ID
Veeam Backup & Replication reports
The first alarm is one that can trigger in the production data sets. Veeam ONE also has advanced logic looking at the backup infrastructure. I recommend taking a similar approach with these 3 alarms in the Backup & Replication section of Veeam ONE’s alarms:
- Immutability change tracking
- Immutability state
- Suspicious incremental backup size
These three alarms monitor key aspects of the backups that are taken having an abnormal change rate (Suspicious incremental backup size) and any changes to the immutability configuration of backup repositories (including turning it off). The figure below shows the conditions of the immutability change tracking alarm:
I recommend for these three alarms, at a minimum as well having email go to two or more persons as well as a script that will run a more pointed task, such as logging an event in your SIEM platform. Simply having the Veeam ONE console flag the alarm and nobody see it or do anything with that information would be a miss.
Veeam Backup & Replication alarms
My last recommendation is to take a key Veeam ONE report and have it automatically sent to backup administrators as well as the security teams. The Restore Operator Activity report is a by far my favorite report, but also one that can be a very critical indicator of data moving elsewhere. This is an alternative view of when data is restored; not just in the case of intended recovery actions. But what if someone restored data elsewhere, and not for an outage. One could call that exfiltration. Below is a sample view of a month’s worth of restore operator activity from a report.
One question that comes to me is why the rick.vanover username did restores with two different accounts. Is that planned, is that a violation? The subsequent pages of this report do a good job of explaining WHAT was restored by each session. My recommendation is that this report be automatically sent via email, which is a very easy task to do in the Veeam ONE Web Client.
Extra – behavior of a possible attacker
Perhaps we can talk about the behavior of attackers, were they stay inside the infrastructure for weeks if not months, and that having the next Report at hand:
And make sure to take a look weekly, or at least monthly, will show the changes to the Backup Infrastructure. Perhaps a new user has been added as an admin, maybe a new repository, or object storage repo where the attackers want to backup copy data, etc.
Similar to that, thinking on the smart attackers that infiltrate an organization and stay there for a long period of time, Veeam offers a great insight on what is happening into VMware vSphere as well:
What do you think? In the end, the result is the same - restore data that has been backed up. Secure Backup is the last line of defense; and Veeam ONE can tell you what is going on to indicate unexpected behavior!
I had a chance to ask around also - and the Backup state alarms, VM Guest Disk Space, Repository Free Space are other key good alarms to handle in the same manner! Thanks for this tip
Oh I should also note that
@jorge.delacruz helped on the review and contribution here!
Really great read Rick. Need to take a look at this alarm in VONE and leverage it in Production for sure.
In my “small” environment, I would love to implement it!
always running out of time to do so.
I will write it down in the things to do!
ransomware stills being my daily nightmare…
Have a good one.
Very interesting article and very useful alarms.
I went back and forth on the “VM with no backup” alarm and “Computer with no backup” alarms - as you can’t recover what you didn’t back up. So I’d consider those as highly recommended to have alarms that both notify and use the pre-built remediation action to put them into a backup job automatically.
@Rick Vanover ! Besides the alarms that indicate attacks that have just been carried out, I especially like the infrastructure reports. They help significantly to detect intruders before they carry out attacks.
I would also add the “Failed login attempt” to VMware or Hyper-V hosts as an alarm that could potentially be treated with escalation with Veeam ONE!
Really useful post
@Rick Vanover! It's much better to discover possible threat actors proactively, then calling the firefighters afterwards.
Has anyone ever implemented some scripted traps which, for example, shutdown the backup systems in case if any suspicious actions?
I have a new addition to this advice. I recommend taking the Backup Alarms report (maybe copy it) and filtering it to just these two alarms:
Then have this report run on a schedule and go directly to security teams and backup administrators. Here is a sample of this report filtered to just these alarms.