ReFS issues with latest Windows Server Updates (KB5009624, KB5009557, KB5009555)


Userlevel 7
Badge +6

Just a quick post; the latest Windows Server updates for 2012R2, 2019 and 2022 (haven’t seen 2016) can cause ReFS issues. After the installation, ReFS volumes are shown as RAW and are no longer accessible; so if you’re using ReFS for your repositories, then take be aware of this when installating the update. Besides that, those updates can also cause bootloops for domain controllers and break Hyper-V.

Depending on your Windows Server version one of the following updates could have caused the issue: KB5009624, KB5009557, KB5009555

If you’re affected then removing the mentioned update should solve the issue. Don’t (!) try to repair the ReFS volume, because this could cause a dataloss.

This should remind everyone why 3-2-1 for Backups is so important. :wink:

Further information:

https://www.bleepingcomputer.com/news/microsoft/new-windows-server-updates-cause-dc-boot-loops-break-hyper-v/

https://forums.veeam.com/veeam-backup-replication-f2/beware-possible-raw-refs-volumes-after-installing-january-updates-t78634.html

Update #1:

Microsoft has pulled the updates, thanks @Mildur 

https://www.bleepingcomputer.com/news/microsoft/microsoft-pulls-new-windows-server-updates-due-to-critical-bugs/amp/

Update #2:

Microsoft has released the updates again and it looks like they didn't fix them. (Thanks @MicoolPaul )

https://www.bleepingcomputer.com/news/microsoft/microsoft-resumes-rollout-of-january-windows-server-updates/

Update #3:

Microsoft has released an out-of-band update, which can possibly resolve the issues:

https://www.bleepingcomputer.com/news/microsoft/microsoft-releases-emergency-fixes-for-windows-server-vpn-bugs/

 


94 comments

Userlevel 7
Badge +5

ReFS issues and DC bootloop and broken Hyper-v

Just talking about this some minutes ago.

More info here: https://www.bleepingcomputer.com/news/microsoft/new-windows-server-updates-cause-dc-boot-loops-break-hyper-v/

 

Userlevel 7
Badge +4

It’s a good start in 2022. To many things go sideways. :)

Userlevel 7
Badge +5

In Microsoft must change devs, they've been doing everything wrong lately.

Userlevel 7
Badge +4

In Microsoft must change devs, they've been doing everything wrong lately.

Not everything. Problem is, everybody talks only about the bad things :)

 

Userlevel 7
Badge +6

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

In Microsoft must change devs, they've been doing everything wrong lately.

The problem aren’t the devs but rather the changed/missing quality control/assurance. This is an organizational problem, which the devs probably can’t do much about. :no_mouth:

Userlevel 7
Badge +4

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Userlevel 7
Badge +8

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆

Userlevel 7
Badge +4

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆

haha :D

Userlevel 7
Badge +5

I can confirm that at the point in writing this piece, Microsoft hasn't recognised this vulnerability

Userlevel 7
Badge +5

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

But too many bad things this month, such as

  • Windows KB5009543, KB5009566 updates break L2TP VPN
  • New critical Windows HTTP vulnerability is wormable etc…

Therefore, I agree with @regnor assertion on this issue! but like I said before, they are still not recognised by Microsoft at this moment of writing this comment: https://msrc.microsoft.com/update-guide/ 

Userlevel 7
Badge +7

Thanks for sharing @regnor! Really bad bugs that shouldn't exist! 

These days a customer told me he was suffering from regular reboots of his new 2022 test server. The guess was it had to do with trial-version. No its a feature … eh … bug

Userlevel 7
Badge +6

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

 You’re definetly right with that :wink:

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆

Better check with a different person, just to be sure :wink:

But perhaps it’s a security feature? Microphones and webcams can spy on you, so better throw it all away :laughing:

@Iams3leI’m sure this won’t be the last time we see such issues; just hope it won’t be in January…

@vNote42Reboots tend to solve problems and make the system more stable; so you’re spot on with “it’s a feature” :nerd:

Userlevel 7
Badge +8

Thanks for sharing. Passed this along to our ops team as today is patching day. Make sure these are not applied to our ReFS servers that are repos. Good old MS.

Userlevel 7
Badge +4

Microsoft has pulled now the updates from the update servers:

https://www.bleepingcomputer.com/news/microsoft/microsoft-pulls-new-windows-server-updates-due-to-critical-bugs/amp/

Userlevel 7
Badge +8

Think it’s for the best… 😬

Userlevel 7
Badge +10

Windows ah windows… Well apart from the this REFS fun (remember the REFS issues when it first came out and those poor souls who ventured into the 4k block settings), I believe there was some completely data wipeouts there. Huge thread in the forums if I remember correctly. That aside can you believe that you need to do this to get NFS write permissions on a nfs share? Or have I really gotten confused :)

“Browse to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ClientForNFS\CurrentVersion\Default.
Create a new New DWORD (32-bit) Value inside the Default folder named AnonymousUid and assign the UID found on the Linux directory as shared by the NFS system. This is the UID of the user that has the write access to that directory on Linux system.
Create a new New DWORD (32-bit) Value inside the Default folder named AnonymousGid and assign the GID found on the Linux directory as shared by the NFS system. This is the GID of the group that has the write access on the directory on Linux system.Windows 10: Regedit NFS AnonymousUid and AnonymousGid
Restart the NFS client or reboot the machine to apply the changes.
Now run the mount command and you will get the write access.”

Userlevel 7
Badge +4

aww bad patch MS

//VPN /IPSec Issue: Certain IPSEC connections might fail

 

//Domain Controller rebooting issue

 

//Hyper V issue

Userlevel 4

KB5009555 for Windows 2022 doesn’t seem to have been pulled. My test server still downloads this update from Windows Update. However, this is the faulty update which causes the raw ReFS issue.

Userlevel 4
Badge

We decided to decline the updates from our WSUS after seeing the thread on Reddit before MS pulled them. 
https://www.reddit.com/r/sysadmin/comments/s1jcue/patch_tuesday_megathread_20220112/
 

The servers that got patched don’t show any errors.

Userlevel 7
Badge +8

KB5009555 for Windows 2022 doesn’t seem to have been pulled. My test server still downloads this update from Windows Update. However, this is the faulty update which causes the raw ReFS issue.

That’s because Microsoft released them AGAIN on a FRIDAY 🤯

 

https://www.bleepingcomputer.com/news/microsoft/microsoft-resumes-rollout-of-january-windows-server-updates/

Userlevel 7
Badge +8

All they’ve done is add the problems as known issues… oh Microsoft…

Userlevel 7
Badge +5

All they’ve done is add the problems as known issues… oh Microsoft…

You are right: https://support.microsoft.com/en-us/topic/january-11-2022-kb5009557-os-build-17763-2452-c3ee4073-1e7f-488b-86c9-d050672437ae

Userlevel 7
Badge +6

All they’ve done is add the problems as known issues… oh Microsoft…

What else should they do...fix the update?🤣

Userlevel 4

But no mention of the ReFS issues. Sigh….

Userlevel 7
Badge +8

All they’ve done is add the problems as known issues… oh Microsoft…

What else should they do...fix the update?🤣

That’s crazy talk! What next, QA testing?

Comment