Skip to main content

My console stopped opening yesterday, crashing and complaining about clr.dll.

 

There’s a R&D post about removing the user.config file (Here), but that didn’t work.

 

Anyone else recently seeing this?

 

Log Name:      Application
Source: Application Error
Date: 12/7/2023 6:27:37 AM
Event ID: 1000
Task Category: (100)
Level: Error
Keywords: Classic
User: N/A
Computer: MANBK-CARM-VEM
Description:
Faulting application name: veeam.backup.shell.exe, version: 12.1.0.2131, time stamp: 0xf82aa70a
Faulting module name: clr.dll, version: 4.8.4645.0, time stamp: 0x648f6f63
Exception code: 0xc0000005
Fault offset: 0x0000000000018551
Faulting process id: 0x2064
Faulting application start time: 0x01da290041bfb33e
Faulting application path: C:\Program Files\Veeam\Backup and Replication\Console\veeam.backup.shell.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll
Report Id: 53fe947b-0942-450e-9476-4cda770d3a62
Faulting package full name:
Faulting package-relative application ID:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Application Error" />
<EventID Qualifiers="0">1000</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>100</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2023-12-07T11:27:37.5902178Z" />
<EventRecordID>108406</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>Application</Channel>
<Computer>MANBK-CARM-VEM</Computer>
<Security />
</System>
<EventData>
<Data>veeam.backup.shell.exe</Data>
<Data>12.1.0.2131</Data>
<Data>f82aa70a</Data>
<Data>clr.dll</Data>
<Data>4.8.4645.0</Data>
<Data>648f6f63</Data>
<Data>c0000005</Data>
<Data>0000000000018551</Data>
<Data>2064</Data>
<Data>01da290041bfb33e</Data>
<Data>C:\Program Files\Veeam\Backup and Replication\Console\veeam.backup.shell.exe</Data>
<Data>C:\Windows\Microsoft.NET\Framework64\v4.0.30319\clr.dll</Data>
<Data>53fe947b-0942-450e-9476-4cda770d3a62</Data>
<Data>
</Data>
<Data>
</Data>
</EventData>
</Event>
Log Name:      Application
Source: .NET Runtime
Date: 12/7/2023 6:27:36 AM
Event ID: 1023
Task Category: None
Level: Error
Keywords: Classic
User: N/A
Computer: MANBK-CARM-VEM
Description:
Application: veeam.backup.shell.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an internal error in the .NET Runtime at IP 00007FFE567C8551 (00007FFE567B0000) with exit code 80131506.

Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name=".NET Runtime" />
<EventID Qualifiers="0">1023</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime="2023-12-07T11:27:36.0782195Z" />
<EventRecordID>108405</EventRecordID>
<Correlation />
<Execution ProcessID="0" ThreadID="0" />
<Channel>Application</Channel>
<Computer>MANBK-CARM-VEM</Computer>
<Security />
</System>
<EventData>
<Data>Application: veeam.backup.shell.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an internal error in the .NET Runtime at IP 00007FFE567C8551 (00007FFE567B0000) with exit code 80131506.
</Data>
</EventData>
</Event>

 

Looks like you just upgraded to v12.1?

I've not had this issue myself. Also, by what your error shows, you may have an issue with your .NET install. Maybe try downloading it and reinstalling? I assume a Console repair didn't work? 


The DLL file you posted is related to .NET for sure so I would investigate that first to ensure it is updated to the most current.  Also which version of Windows are you installing the console on?  Maybe remove the console and address .NET then reinstall it.


Is this the local console on the Veeam Server or is it a remote console on another server?

If it is the second, did you update the remote console, too?


Is this the local console on the Veeam Server or is it a remote console on another server?

If it is the second, did you update the remote console, too?

Ah yes version mismatch.  Good catch Joe both ends need to be the same. 👍🏼


@

Looks like you just upgraded to v12.1?

I've not had this issue myself. Also, by what your error shows, you may have an issue with your .NET install. Maybe try downloading it and reinstalling? I assume a Console repair didn't work? 

I actually upgraded to 12.1 just to see if this would go away.  No help.

 

The DLL file you posted is related to .NET for sure so I would investigate that first to ensure it is updated to the most current.  Also which version of Windows are you installing the console on?  Maybe remove the console and address .NET then reinstall it.

I uninstalled and reinstalled .net in server 2022.  Still crashes.

 

Is this the local console on the Veeam Server or is it a remote console on another server?

If it is the second, did you update the remote console, too?

This is on the local server.  until now, I refused to allow a remote console, now I’m setting up a PAW to see if a second machine will at least open the console.

 

 


Here’s an update:

After installing the Console on a Windows 11 23H2 PAW, it opens and connects to the backup server.

The remote console displays an ‘Update Compnents’ panel after my upgrade to 12.1.

The problem seems to be running the console on the Server.  It went from working fine at 0700, to crashing at 0800 with no reconfigurations/updates to the server.

Still not sure why.

If I fine anything more useful than the errors above, I’ll post it for community reference.


I have the same issue, the console is keep crashing.

Any solution for so far?


I have the same issue, the console is keep crashing.

Any solution for so far?

None.  Just been using a remote console on a PAW.  Very annoying.  I guess we’ll do that until another update is released of either the Veeam product, or another .net update.


Ugh...well, please keep us posted on the solution you come up with @Cory 


Hello Guys,

I have setup a new Backup server with the latest Veeam edition. Imported the configuration to the new Veeam server and the Problem is solved. Everything work Perfect for so far. Just rescan the respos en enable the jobs. Thats it.

 

 


Nice! A bit of work, but glad you got yours sorted @ki.student 


Hello Guys,

I have setup a new Backup server with the latest Veeam edition. Imported the configuration to the new Veeam server and the Problem is solved. Everything work Perfect for so far. Just rescan the respos en enable the jobs. Thats it.

 

 

Great to see you got this working.  Hopefully this helps the OP with his issue. 👍🏼


Hello,

we had the same problem and we tried everything to discover that the problem was caused by our Antivirus, McAfee/Trellix adaptive threat prevention latest update, making exclusions in epo made everything work fine again.


@abdallah.rajab 

Can you please provide a list of the exceptions you had to implement? I have run into the exact same issue and have confirmed that Trellix/McAfee ATP is the culprit.


Here is the main ones for Veeam - https://www.veeam.com/kb1999

 


Thank you very much for the response. What fixed it for me was this, McAfee/Trellix Issues.

Its a documented issue with 32-bit applications and McAfee/Trellix ATP after their September 2023 Endpoint Security Update. Hope this helps others who encounter this issue.

 

 


Hey @Cory -

Did you ever get your Console to open on your server? Just following up to see what you did. Did you reach to Support? If they helped, can you share their solution for you?

Thanks.


@coolsport00 Thanks for bringing me back to this issue.  I just saw the comments about Trellix ATP by @abdallah.rajab. Thanks both of you.

 

I also disabled ATP and proved it worked.  I searched the Trellix logs so many times and didn’t see that.

@Lorenzo.Ramirez, thanks for that KCM.  I was super annoyed I didn’t see it with all the googling I did, but then I noticed the article date and after I initially had this issue.

 

I’ll build and ATP exception and be good to go.


Great to hear! 


Great to hear you fixed it Cory.


The source IS not Mcafee/Trellix you JUST ignore the ERROR that comes from MS Windows Defender with that option:

 

It's crucial to note that issues related to Mcafee or Trellix are not the root cause here.

Contrary to common belief, the issue is not directly linked to Mcafee or Trellix. Instead, it stems from the AMSI module. Trellix, like many other antivirus (AV) solutions, relies on AMSI from Windows Defender to effectively scan PowerShell scripts. Therefore, the error is inherently tied to the fact that Trellix, as with any sophisticated AV enterprise solution, leverages components of Windows Defender.

It's worth emphasizing that even if you have Mcafee or Trellix Endpoint Security (ENS) installed, certain aspects of Microsoft Windows Defender remain operational.

The Windows Antimalware Scan Interface (AMSI) serves as a crucial interface standard, enabling seamless integration between your applications and services with any antimalware product installed on a machine. AMSI plays a pivotal role in providing advanced malware protection, safeguarding end-users, their data, applications, and workloads.

Is someone still using Russian 32BIT .NET Code in 2024? Oh yes the company who want’s us ALSO to protect us from ransomware…. 😉 Guess which one?

32-bit applications crashing due to .NET runtime error after installing Endpoint Security 10.7 September 2023 Update (trellix.com)

https://kcm.trellix.com/corporate/index?page=content&id=KB96789

Hopes this helps some of you….


Thanks for the further clarification @BackupExec 


Comment