Veeam's Malware Detection feature released with v12.1 was certainly a much-anticipated and welcomed feature by many, me included. And, if you didn't read it, I even wrote about my initial experiences with it:

But since enabling Inline Entropy Scan in my environment, I've really had to micro-manage it with regards to Malware events of potential malicious activity Veeam "finds". Veeam is certainly aware of the short-comings of the Inline Scan engine and I know they are hard at work preparing an update for release which I'm confident will make it better. See this Forums post for further details.

In the meantime, I've had to really dig into doing a lot of Scan Backup and manual scan tasks, as well as get creative in how I do forensics on my systems to make sure the Malware Detection events I'm receiving are false positives. And if not, take care of those issues appropriately as well. I thought I'd share some of the forensics and analyses processes I'm doing in my environment to help the Community when you also receive Malware events. Again, the processes I share below are with regard to detections via Inline Entropy scans because not much data is provided by these events. Some of what I show how to do below can of course also be applied for File System Analysis (FSA) scan events, but some of the tweaking you can do with FSA is currently not available for Inline Scans.

You Received A Malware Event Notification

Regardless which of the 3 Inline Scan event type notifications you receive – Encrypted Files, Onion Links, or Ransomware Note – there just isn't much information Veeam provides to help you find the culprit file(s)…if there indeed is any. The event for each looks very similar to the following:

To attempt to get more detail, there is no dedicated log for the VM and event distinctly. Rather, all Inline Entropy Scan events are noted in a single log file located at: C:\ProgramData\Veeam\Backup\Svc.VeeamDataAnalyzer.log . To date, I've not seen a potential "culprit" file or directory listed in this log file, sadly. I still think it's worth an initial glance at though. So, what should you do next?
 

Scans

A/V Scan

Veeam to the rescue! Or, more specifically, the Veeam Mount Server to the rescue, where Veeam keeps Antivirus configuration XML files so you can scan your systems with your organization's Antivirus tool.  After receiving an Inline Entropy Malware event notification and reviewing the event log file, go into the Veeam Console to see which VMs were flagged as Suspicious. My suggestion to start is go into the Home node, then under Backups click on Disks, then expand a Job marked with a red Malware icon. Right-click your potentially infected VM, then choose the Scan Backup option. NOTE: currently, this task can only be done on Windows VMs, but scanning for Linux systems will be available soon, per Product Managers.

To start with, just perform a simple A/V scan, and I suggest doing so for a Date Range to only go back as far as the 1st time you received the Malware event notification..plus one day, to make sure backups for the VM before the event are good.

If you want to do multiple A/V scans to get through your Malware forensics a bit quicker, be aware you can only run one A/V scan at a time in the Veeam Console. Well, let me clarify this – I was only able to run 1 A/V scan at a time because I have only 1 Mount Server configured for all my Repositories. Don't quote me on this, but if you have different Mount Servers configured for all your Repositories and your Malware event VMs are on different Repositories, I would think you should be able to run multiple A/V scans at a time since the Mount Server is used for this Scan Backup task. I'll try and get confirmation on this and update this post.

Otherwise, if your environment is configured like mine to where you're not able to run multiple A/V scans at once within the Console, you'll need to run them manually directly on the VM. If your environment runs the default Microsoft Windows Defender antivirus solution, keep in mind it wasn't a part of Windows OS until Windows Server 2016. If you have Windows 2012 R2 or earlier (eek!), you can download and run a free Microsoft scan tool: Microsoft Safety Scanner . 

YARA Scan

A "what-uh"? That would be YARA, which stands for "Yet Another Recursive Acronym". In security terms, it's a means by which to perform a malware scan on a system using rule-based .yar files. After you run an A/V scan on your VMs, I recommend running a YARA scan. These scans can be ran in the Veeam Console when you perform a Scan Backup operation. But, instead of selecting to perform an Antivirus scan, select YARA, then choose a Rule from the drop-down:

If there's a specific piece of malware you're wanting to scan for, search for the rule on the Internet and put it in a text file, changing the file extension to .yara. To use it in Veeam, just place the file in the following directory for use:
C:\Program Files\Veeam\Backup and Replication\Backup\YaraRules

You should then see it from the drop-down menu.

Now, that's all well and good, but what if you have Veeam Data Protection Foundation Edition like me; if so, you won't be able to do YARA scans in Veeam. You must have either Advanced (with VeeamONE) or Premium (with VeeamONE and Recovery Orchestrator) Editions. So now what? Well, ok…so you don't have a central Console to run YARA scans from. But, you can still run a YARA scan directly within your VM's OS just like you did with A/V. Just download the Windows YARA binaries, unzip the file, and place both the yara64.exe and yarac64.exe files in a folder. Now all you need is a YARA rules file in the same folder, with a .yar extension. I'll give you a sample one to start. To run a scan for Onion Links, copy & paste the below code in a text file and change the file extension:

rule OnionLinks

{

    meta:

        description = “Onion Link”

    strings:

        $onion_link = /\b(
    condition:

        $onion_link

}

Install for Linux – as you'd expect, this is a bit different. For Ubuntu, either from the VM Console or via SSH, enter the following command to install the YARA tool:

sudo apt insatll yara

You can use the same YARA rule file for Linux as for Windows. Running a YARA scan in Linux is similar to Windows. I'll mostly go into detail here with Windows, but similar to Windows scanning, create a YARA rule file then run a YARA scan against a file or directory:

yara /my/path/rule.yar /directory/to/scan

See here for more command line help.

To scan your Windows system, I recommend using Powershell. Enlarge the PoSH Window so you can better see the full file path lines in the output, change your directory to the path you placed the YARA files in, then type in the following line:

PS C:\Path-to-YARA\.\yara64.exe onionlinks-rule.yar -r C:\

Let me briefly explain what the above command does. You are running a YARA scan (yara64.exe), with the .yar file you created (onionlinks-rule.yar) recursively (-r)  against your C:\ drive. If you want to scan only in a certain folder, you can do that too (e.g. C:\Path\MyFolder). After you type in the command, press <ENTER> and away it goes. You'll see a lot of error scanning lines, which can safely be ignored. What you're interested in are onion links lines, like the ones shown below:

Take note, the executable Onion Links shown above are false positives because they are a part of Windows. And here lies another shortfall of the Inline Entropy Scan engine – no ability to exclude files or directories like you can in the File System Analysis engine.  Again, hopefully this gets addressed in an upcoming update. But again, to Veeam’s credit, the nature of how these scans run, which is at the block level, granularity like this feature request may not be possible.

SEARCH TIP: To be able to more readily find Onion Link lines from the output, I recommend going to the beginning where you initially typed the YARA command, then click and drag to select all the output down to the last line. Then, copy and paste the output into a text editor of your choice. Simply do a "Find" operation to find all lines with the word Onion in it.

Remediation

When you run your A/V scan, at least directly on the VM, it'll usually quarantine and remove critical malicious files. Some A/V engines allow you granularity to select what to do if it finds such files – either quarantine, remove them, or even leave them alone. Verify if the file(s) are valid or not then remove any files as needed.

But hopefully, it finds no such files and all is clean. For Veeam-ran A/V scans, you can review the log file of the scan from the Mount Server to see what was found, then take any appropriate action. The log is located at:

C:\ProgramData\Veeam\Backup\FLRSessions\Windows\<FLR_VM-name_>\Antivirus\<av-name>-Volume#.log

Next, look at your YARA scan output (again, this is if you ran this directly on your VM). As I shared above, copy/paste the output from the scan in a text editor then find any Onion Link lines. Verify whether the file(s) found in the given path(s) are legitimate or not. If not, simply remove them.

After you've performed the above forensics, analyses, & remediation steps, you then need to go back on your Veeam Server and begin marking your VMs as “Clean” or not based off what you've found. Here is another shortfall of the Inline Entropy (and even FSA) Scan engine – if after your investigation you choose to mark a restore point as "Clean", any files you left on the VM (because, for example, you determined they were not malicious), subsequent scans by Veeam will again detect them and mark them as Suspicious, causing yet another Malware event notification. It would be nice for Veeam to allow subsequent scans to still be "Clean" as you just marked. In other words, if there are still files in the same path(s) as before, but based off your analysis are false positives, Veeam should not mark future Restore Points as Suspicious. Rather, Veeam should only do so if there are new Suspicious files. Maybe adding Exclusions would resolve this? 

Repeat the processes above for subsequent VMs as needed.

Summary

Hopefully the above process will give you a good starting point on how to begin tackling Malware events you receive from Veeam. As you can see, Veeam certainly has some tweaking to do in this feature. But, in my opinion, it's still a viable tool to use in your security arsenal for your backup environment. It helped me detect a virus in my enviroment. 

If you have any other suggestions on how you handle such events, I welcome any comments you have below.

Cheers!

Resources:
Veeam User Guide
YARA documentation