Hi Community!
If you use Veeam Backup & Replication to protect Linux environments, you may have encountered the strict requirements of its kernel modules. I recently had an interesting problem with a Oracle Linux 8.9 system running Unbreakable Enterprise Kernel (UEK).
Situation
The plan was simple: use the Veeam Agent for Linux (VAL) in managed mode to backup some Folders and a PostgreSQL database. I needed something called "Application-Aware Processing" (AAP) to make sure the database was consistent, so I couldn't use 'nosnap mode' (which doesn't require the kernel module).
Task
Here is what the environment looked like:
-
Veeam Agent for Linux (VAL): 6.3.2
-
OS: Oracle Linux 8.9
-
Kernel (UEK): 5.15.0-205.149.5.1.el8uek.x86_64
- Veeam module status: veeamsnap, 6.3.2.1207: added (via dkms status)
Issue
When starting the backup job from the VBR server, the task fails, returning this sequence of errors:
- Failed to create volume snapshot
- Failed to take volume snapshot
- Failed to perform managed backup
- Veeam snapshot kernel module not found.
Reading the official KB2804, regarding kernel compatibility, it seemed like the VAL version and the kernel were simply not communicating correctly.
As suggested in the note, I opened the Veeam KB4394, which deals exactly with compiling the veeamsnap module for UEK kernels using an updated GCC toolset (in my case, gcc-toolset-11).
But when I tried to follow the steps and force the compilation, the process kept failing, throwing a compiler mismatch warning:
warning: the compiler differs from the one used to build the kernel The kernel was built by: gcc (GCC) 11.4.1 20230605 (Red Hat 11.4.1-2.1.0.1) You are using: gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-20.0.3)
I had enabled the scl environment with GCC 11, but the background build process kept falling back to the system's default GCC 8.5. This meant that the DKMS module failed completely because of some compiler flags that weren't recognised.
Action
I tried to resolve the issue several times, but failed. I then opened a support ticket on Veeam support portal.
The support engineer had a really good point about a detail that's easily missed: the version of the DKMS framework on the system. My machine was using DKMS v2.7.
It looks like this older version of DKMS has had problems with managing environment variables with Software Collections (scl). Basically, it doesn't tell you which compiler to use.
Here are the steps to definitively solve the issue:
-
Update DKMS package from version 2.7 to version 3.x.
-
I followed the instructions in KB4394 to rebuild the module, and it worked perfectly first time. The veeamsnap module was successfully compiled using GCC 11, with no errors that could not be solved.
Result
After restarting the Veeam service on the Linux server, the VBR job launched successfully, the volume snapshots were created, and the backup completed without a single error.
If you're having trouble with GCC mismatches when you're compiling the veeamsnap module for UEK kernels, and KB4394 isn't working, take a step back. Before you spend too long, check your DKMS version. Upgrading to version 3 could save you hours of troubleshooting!
See you next time!
