Skip to main content

ReFS issues with latest Windows Server Updates (KB5009624, KB5009557, KB5009555)


regnor
Forum|alt.badge.img+14

Just a quick post; the latest Windows Server updates for 2012R2, 2019 and 2022 (haven’t seen 2016) can cause ReFS issues. After the installation, ReFS volumes are shown as RAW and are no longer accessible; so if you’re using ReFS for your repositories, then take be aware of this when installating the update. Besides that, those updates can also cause bootloops for domain controllers and break Hyper-V.

Depending on your Windows Server version one of the following updates could have caused the issue: KB5009624, KB5009557, KB5009555

If you’re affected then removing the mentioned update should solve the issue. Don’t (!) try to repair the ReFS volume, because this could cause a dataloss.

This should remind everyone why 3-2-1 for Backups is so important. :wink:

Further information:

https://www.bleepingcomputer.com/news/microsoft/new-windows-server-updates-cause-dc-boot-loops-break-hyper-v/

https://forums.veeam.com/veeam-backup-replication-f2/beware-possible-raw-refs-volumes-after-installing-january-updates-t78634.html

Update #1:

Microsoft has pulled the updates, thanks @Mildur 

https://www.bleepingcomputer.com/news/microsoft/microsoft-pulls-new-windows-server-updates-due-to-critical-bugs/amp/

Update #2:

Microsoft has released the updates again and it looks like they didn't fix them. (Thanks @MicoolPaul )

https://www.bleepingcomputer.com/news/microsoft/microsoft-resumes-rollout-of-january-windows-server-updates/

Update #3:

Microsoft has released an out-of-band update, which can possibly resolve the issues:

https://www.bleepingcomputer.com/news/microsoft/microsoft-releases-emergency-fixes-for-windows-server-vpn-bugs/

 

106 comments

marcofabbri
Forum|alt.badge.img+13
  • On the path to Greatness
  • 990 comments
  • January 13, 2022

ReFS issues and DC bootloop and broken Hyper-v

Just talking about this some minutes ago.

More info here: https://www.bleepingcomputer.com/news/microsoft/new-windows-server-updates-cause-dc-boot-loops-break-hyper-v/

 


Mildur
Forum|alt.badge.img+12
  • Influencer
  • 1035 comments
  • January 13, 2022

It’s a good start in 2022. To many things go sideways. :)


marcofabbri
Forum|alt.badge.img+13
  • On the path to Greatness
  • 990 comments
  • January 13, 2022

In Microsoft must change devs, they've been doing everything wrong lately.


Mildur
Forum|alt.badge.img+12
  • Influencer
  • 1035 comments
  • January 13, 2022
marcofabbri wrote:

In Microsoft must change devs, they've been doing everything wrong lately.

Not everything. Problem is, everybody talks only about the bad things :)

 


regnor
Forum|alt.badge.img+14
  • Author
  • Veeam MVP
  • 1352 comments
  • January 13, 2022

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

marcofabbri wrote:

In Microsoft must change devs, they've been doing everything wrong lately.

The problem aren’t the devs but rather the changed/missing quality control/assurance. This is an organizational problem, which the devs probably can’t do much about. :no_mouth:


Mildur
Forum|alt.badge.img+12
  • Influencer
  • 1035 comments
  • January 13, 2022
regnor wrote:

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 


MicoolPaul
Forum|alt.badge.img+23
  • 2362 comments
  • January 13, 2022
Mildur wrote:
regnor wrote:

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆


Mildur
Forum|alt.badge.img+12
  • Influencer
  • 1035 comments
  • January 13, 2022
MicoolPaul wrote:
Mildur wrote:
regnor wrote:

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆

haha :D


Iams3le
Forum|alt.badge.img+11
  • Veeam Legend
  • 1394 comments
  • January 13, 2022

I can confirm that at the point in writing this piece, Microsoft hasn't recognised this vulnerability


Iams3le
Forum|alt.badge.img+11
  • Veeam Legend
  • 1394 comments
  • January 13, 2022
Mildur wrote:
regnor wrote:

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

But too many bad things this month, such as

  • Windows KB5009543, KB5009566 updates break L2TP VPN
  • New critical Windows HTTP vulnerability is wormable etc…

Therefore, I agree with @regnor assertion on this issue! but like I said before, they are still not recognised by Microsoft at this moment of writing this comment: https://msrc.microsoft.com/update-guide/ 


vNote42
Forum|alt.badge.img+13
  • On the path to Greatness
  • 1246 comments
  • January 13, 2022

Thanks for sharing @regnor! Really bad bugs that shouldn't exist! 

These days a customer told me he was suffering from regular reboots of his new 2022 test server. The guess was it had to do with trial-version. No its a feature … eh … bug


regnor
Forum|alt.badge.img+14
  • Author
  • Veeam MVP
  • 1352 comments
  • January 13, 2022
Mildur wrote:
regnor wrote:

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

 You’re definetly right with that :wink:

MicoolPaul wrote:
Mildur wrote:
regnor wrote:

@Mildur But so far we havn’t experienced any good things since 12/31/2021 from them; except you count everything which is not broken or doesn’t have issues as positive :wink:

 

True. I was talking more general. Not only in 2022, and not only from Microsoft.

Bad things are more discussed than good things :)

 

Nope that’s it, Microsoft took it too far this morning! Decided my headset and webcam were untrustworthy (or maybe they meant the person on them, aka me!) 😆

Better check with a different person, just to be sure :wink:

But perhaps it’s a security feature? Microphones and webcams can spy on you, so better throw it all away :laughing:

@Iams3leI’m sure this won’t be the last time we see such issues; just hope it won’t be in January…

@vNote42Reboots tend to solve problems and make the system more stable; so you’re spot on with “it’s a feature” :nerd:


Chris.Childerhose
Forum|alt.badge.img+21
  • Veeam Legend, Veeam Vanguard
  • 8506 comments
  • January 13, 2022

Thanks for sharing. Passed this along to our ops team as today is patching day. Make sure these are not applied to our ReFS servers that are repos. Good old MS.


Mildur
Forum|alt.badge.img+12
  • Influencer
  • 1035 comments
  • January 13, 2022

MicoolPaul
Forum|alt.badge.img+23
  • 2362 comments
  • January 13, 2022
Mildur wrote:

Think it’s for the best… 😬


Geoff Burke
Forum|alt.badge.img+22
  • Veeam Legend, Veeam Vanguard
  • 1318 comments
  • January 14, 2022

Windows ah windows… Well apart from the this REFS fun (remember the REFS issues when it first came out and those poor souls who ventured into the 4k block settings), I believe there was some completely data wipeouts there. Huge thread in the forums if I remember correctly. That aside can you believe that you need to do this to get NFS write permissions on a nfs share? Or have I really gotten confused :)

“Browse to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ClientForNFS\CurrentVersion\Default.
Create a new New DWORD (32-bit) Value inside the Default folder named AnonymousUid and assign the UID found on the Linux directory as shared by the NFS system. This is the UID of the user that has the write access to that directory on Linux system.
Create a new New DWORD (32-bit) Value inside the Default folder named AnonymousGid and assign the GID found on the Linux directory as shared by the NFS system. This is the GID of the group that has the write access on the directory on Linux system.Windows 10: Regedit NFS AnonymousUid and AnonymousGid
Restart the NFS client or reboot the machine to apply the changes.
Now run the mount command and you will get the write access.”


Link State
Forum|alt.badge.img+11
  • Veeam Legend
  • 612 comments
  • January 14, 2022

Forum|alt.badge.img
  • Comes here often
  • 29 comments
  • January 14, 2022

KB5009555 for Windows 2022 doesn’t seem to have been pulled. My test server still downloads this update from Windows Update. However, this is the faulty update which causes the raw ReFS issue.


vergoted
Forum|alt.badge.img+1
  • Comes here often
  • 24 comments
  • January 14, 2022

We decided to decline the updates from our WSUS after seeing the thread on Reddit before MS pulled them. 
https://www.reddit.com/r/sysadmin/comments/s1jcue/patch_tuesday_megathread_20220112/
 

The servers that got patched don’t show any errors.


MicoolPaul
Forum|alt.badge.img+23
  • 2362 comments
  • January 15, 2022
Franc wrote:

KB5009555 for Windows 2022 doesn’t seem to have been pulled. My test server still downloads this update from Windows Update. However, this is the faulty update which causes the raw ReFS issue.

That’s because Microsoft released them AGAIN on a FRIDAY 🤯

 

https://www.bleepingcomputer.com/news/microsoft/microsoft-resumes-rollout-of-january-windows-server-updates/


MicoolPaul
Forum|alt.badge.img+23
  • 2362 comments
  • January 15, 2022

All they’ve done is add the problems as known issues… oh Microsoft…


Iams3le
Forum|alt.badge.img+11
  • Veeam Legend
  • 1394 comments
  • January 15, 2022
MicoolPaul wrote:

All they’ve done is add the problems as known issues… oh Microsoft…

You are right: https://support.microsoft.com/en-us/topic/january-11-2022-kb5009557-os-build-17763-2452-c3ee4073-1e7f-488b-86c9-d050672437ae


regnor
Forum|alt.badge.img+14
  • Author
  • Veeam MVP
  • 1352 comments
  • January 15, 2022
MicoolPaul wrote:

All they’ve done is add the problems as known issues… oh Microsoft…

What else should they do...fix the update?🤣


Forum|alt.badge.img
  • Comes here often
  • 29 comments
  • January 15, 2022

But no mention of the ReFS issues. Sigh….


MicoolPaul
Forum|alt.badge.img+23
  • 2362 comments
  • January 15, 2022
regnor wrote:
MicoolPaul wrote:

All they’ve done is add the problems as known issues… oh Microsoft…

What else should they do...fix the update?🤣

That’s crazy talk! What next, QA testing?


Comment