Skip to main content

Just a quick post; the latest Windows Server updates for 2012R2, 2019 and 2022 (haven’t seen 2016) can cause ReFS issues. After the installation, ReFS volumes are shown as RAW and are no longer accessible; so if you’re using ReFS for your repositories, then take be aware of this when installating the update. Besides that, those updates can also cause bootloops for domain controllers and break Hyper-V.

Depending on your Windows Server version one of the following updates could have caused the issue: KB5009624, KB5009557, KB5009555

If you’re affected then removing the mentioned update should solve the issue. Don’t (!) try to repair the ReFS volume, because this could cause a dataloss.

This should remind everyone why 3-2-1 for Backups is so important. :wink:

Further information:

https://www.bleepingcomputer.com/news/microsoft/new-windows-server-updates-cause-dc-boot-loops-break-hyper-v/

https://forums.veeam.com/veeam-backup-replication-f2/beware-possible-raw-refs-volumes-after-installing-january-updates-t78634.html

Update #1:

Microsoft has pulled the updates, thanks @Mildur 

https://www.bleepingcomputer.com/news/microsoft/microsoft-pulls-new-windows-server-updates-due-to-critical-bugs/amp/

Update #2:

Microsoft has released the updates again and it looks like they didn't fix them. (Thanks @MicoolPaul )

https://www.bleepingcomputer.com/news/microsoft/microsoft-resumes-rollout-of-january-windows-server-updates/

Update #3:

Microsoft has released an out-of-band update, which can possibly resolve the issues:

https://www.bleepingcomputer.com/news/microsoft/microsoft-releases-emergency-fixes-for-windows-server-vpn-bugs/

 

Unfortunately just finding out about this. Entire ReFS volume appears to be shot even after uninstalling the update. Anyone else run into uninstalling not resolving the issue?

 

***Update - after uninstalling BOTH KB5009624 and KB5009595 our ReFS volumes are back. Running Server 2012 R2 Datacenter


Just had the ReFS issue on Server 2016 with KB5009546.  After install, reboot, target USB repository was RAW.  Uninstalled update and drive is visible again with all data.

 

UGH.  This has been a known issue for a week, why hasn’t MS pulled it?


All they’ve done is add the problems as known issues… oh Microsoft…

the bottom in quality management does not seem to have been reached yet:rage:


Out of curiosity, is anyone getting failed to create snapshot on Hyper-V after the Jan 11 patch?

Boot, login and VMs are fine, but I can’t get any backups to work post-update.


All they’ve done is add the problems as known issues… oh Microsoft…

What else should they do...fix the update?🤣

That’s crazy talk! What next, QA testing?

You say it. We get the updates for free, so the least we can do is giving something back to Microsoft; fixing bugs, QA, …

 


Oh.my.god…. 🤯🤯🤯


All they’ve done is add the problems as known issues… oh Microsoft…

What else should they do...fix the update?🤣

That’s crazy talk! What next, QA testing?


But no mention of the ReFS issues. Sigh….


All they’ve done is add the problems as known issues… oh Microsoft…

What else should they do...fix the update?🤣


All they’ve done is add the problems as known issues… oh Microsoft…

You are right: https://support.microsoft.com/en-us/topic/january-11-2022-kb5009557-os-build-17763-2452-c3ee4073-1e7f-488b-86c9-d050672437ae


All they’ve done is add the problems as known issues… oh Microsoft…


KB5009555 for Windows 2022 doesn’t seem to have been pulled. My test server still downloads this update from Windows Update. However, this is the faulty update which causes the raw ReFS issue.

That’s because Microsoft released them AGAIN on a FRIDAY 🤯

 

https://www.bleepingcomputer.com/news/microsoft/microsoft-resumes-rollout-of-january-windows-server-updates/


We decided to decline the updates from our WSUS after seeing the thread on Reddit before MS pulled them. 
https://www.reddit.com/r/sysadmin/comments/s1jcue/patch_tuesday_megathread_20220112/
 

The servers that got patched don’t show any errors.


KB5009555 for Windows 2022 doesn’t seem to have been pulled. My test server still downloads this update from Windows Update. However, this is the faulty update which causes the raw ReFS issue.


aww bad patch MS

//VPN /IPSec Issue: Certain IPSEC connections might fail

 

//Domain Controller rebooting issue

 

//Hyper V issue


Windows ah windows… Well apart from the this REFS fun (remember the REFS issues when it first came out and those poor souls who ventured into the 4k block settings), I believe there was some completely data wipeouts there. Huge thread in the forums if I remember correctly. That aside can you believe that you need to do this to get NFS write permissions on a nfs share? Or have I really gotten confused :)

“Browse to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ClientForNFS\CurrentVersion\Default.
Create a new New DWORD (32-bit) Value inside the Default folder named AnonymousUid and assign the UID found on the Linux directory as shared by the NFS system. This is the UID of the user that has the write access to that directory on Linux system.
Create a new New DWORD (32-bit) Value inside the Default folder named AnonymousGid and assign the GID found on the Linux directory as shared by the NFS system. This is the GID of the group that has the write access on the directory on Linux system.Windows 10: Regedit NFS AnonymousUid and AnonymousGid
Restart the NFS client or reboot the machine to apply the changes.
Now run the mount command and you will get the write access.”


Think it’s for the best… 😬


Microsoft has pulled now the updates from the update servers:

https://www.bleepingcomputer.com/news/microsoft/microsoft-pulls-new-windows-server-updates-due-to-critical-bugs/amp/


Thanks for sharing. Passed this along to our ops team as today is patching day. Make sure these are not applied to our ReFS servers that are repos. Good old MS.


@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

 You’re definetly right with that :wink:

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆

Better check with a different person, just to be sure :wink:

But perhaps it’s a security feature? Microphones and webcams can spy on you, so better throw it all away :laughing:

@Iams3leI’m sure this won’t be the last time we see such issues; just hope it won’t be in January…

@vNote42Reboots tend to solve problems and make the system more stable; so you’re spot on with “it’s a feature” :nerd:


Thanks for sharing @regnor! Really bad bugs that shouldn't exist! 

These days a customer told me he was suffering from regular reboots of his new 2022 test server. The guess was it had to do with trial-version. No its a feature … eh … bug


@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

But too many bad things this month, such as

  • Windows KB5009543, KB5009566 updates break L2TP VPN
  • New critical Windows HTTP vulnerability is wormable etc…

Therefore, I agree with @regnor assertion on this issue! but like I said before, they are still not recognised by Microsoft at this moment of writing this comment: https://msrc.microsoft.com/update-guide/ 


I can confirm that at the point in writing this piece, Microsoft hasn't recognised this vulnerability


@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆

haha :D


@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆


Comment