ReFS issues with latest Windows Server Updates (KB5009624, KB5009557, KB5009555)

2 years ago
13 January 2022
106 comments
14998 views

Userlevel 7

+14

regnor
Veeam MVP
1280 comments

Just a quick post; the latest Windows Server updates for 2012R2, 2019 and 2022 (haven’t seen 2016) can cause ReFS issues. After the installation, ReFS volumes are shown as RAW and are no longer accessible; so if you’re using ReFS for your repositories, then take be aware of this when installating the update. Besides that, those updates can also cause bootloops for domain controllers and break Hyper-V.

Depending on your Windows Server version one of the following updates could have caused the issue: KB5009624, KB5009557, KB5009555

If you’re affected then removing the mentioned update should solve the issue. Don’t (!) try to repair the ReFS volume, because this could cause a dataloss.

This should remind everyone why 3-2-1 for Backups is so important. :wink:

Further information:

https://www.bleepingcomputer.com/news/microsoft/new-windows-server-updates-cause-dc-boot-loops-break-hyper-v/

https://forums.veeam.com/veeam-backup-replication-f2/beware-possible-raw-refs-volumes-after-installing-january-updates-t78634.html

Update #1:

Microsoft has pulled the updates, thanks @Mildur

https://www.bleepingcomputer.com/news/microsoft/microsoft-pulls-new-windows-server-updates-due-to-critical-bugs/amp/

Update #2:

Microsoft has released the updates again and it looks like they didn't fix them. (Thanks @MicoolPaul )

https://www.bleepingcomputer.com/news/microsoft/microsoft-resumes-rollout-of-january-windows-server-updates/

Update #3:

Microsoft has released an out-of-band update, which can possibly resolve the issues:

https://www.bleepingcomputer.com/news/microsoft/microsoft-releases-emergency-fixes-for-windows-server-vpn-bugs/

Max M. | Systems Engineer @Veeam | VMCE/VMCA | former Legend & Vanguard

Like
Quote
Share

Show first post Hide first post

106 comments

Page 5 / 5

Userlevel 7

+14

regnor
Author
Veeam MVP
1280 comments
2 years ago
18 January 2022

Now that was quick @Franc! Lets hope they’ll also be able to solve the fixed disk problem soon.

Userlevel 7

+13

marcofabbri
Veeam Legend
990 comments
2 years ago
18 January 2022

@regnor , well that was quick with MS support. They confirmed it's also an issue with fixed drives, but the patch from yesterday was only for external drives. They are still working on a fix for fixed drives and he will inform me once the patch is available. He couldn't explain though why Microsoft doesn't mention the issue for fixed drives anywhere, but only for external drives. He confirmed he already had multiple cases where other customers also experience the issue with fixed drives.

No good Microsoft, no good...

Userlevel 7

+20

Chris.Childerhose
Veeam Legend, Veeam Vanguard
6275 comments
2 years ago
25 January 2022

Thanks @MicoolPaul and @marcofabbri ! For me Windows Server 2016 was the first contact with ReFS as Veeam started to integrated it. With 2012R2 I felt that it wasn’t mature enough for production. You’re right that the integrity part is/was an advantage, but without storage spaces you only had the corroption detection.

Yeah we never looked at it until 2016 as well since it was just new in 2012R2. Also by the time they came out with 2016 the product had matured and was much better. Did not like anything below 2016 other than maybe 2008R2. :joy:

Userlevel 7

+13

marcofabbri
Veeam Legend
990 comments
2 years ago
25 January 2022

“In any case if you use NTFS I personally recommend using NTFS instead of ReFS, as ReFS is still immature as a file system as has several bugs.”

Wow.

So @Franc if you uninstall that update you can access to data?

Userlevel 7

+20

Chris.Childerhose
Veeam Legend, Veeam Vanguard
6275 comments
2 years ago
13 April 2022

Apparently, this is still an issue. I had 6 updates automatically applied (KB5011564, KB5011560, KB5010462, KB5010419, KB5010395, KB5009595) I thought I had figured out which one was the culprit but I was mistaken and did not have the time to narrow it down so I removed all 6, rebooted, and the ReFS drive no longer showed as RAW. I do not find any of the KB’s that I listed above as matching the other’s in this thread, has anyone else experienced this and found out which one it was?

We have not applied any of these updates to our ReFS servers due to this problem. I am surprised MS has not addressed it yet and we cannot have our Veeam services down.

We have been employing the same approach as you and these shouldn’t have been applied but someone else had a different idea. I will need to isolate which update caused it, thanks for the response.

No problem - we use a patch management system so we don’t approve those updates for the specific servers so they never get installed.

Userlevel 7

+20

Chris.Childerhose
Veeam Legend, Veeam Vanguard
6275 comments
2 years ago
25 January 2022

Latest update from MS Support: case is being archived with no resolution at the moment. Advice currently is, if the issue occurs after installing the cumulative patch, you have two options:

- uninstall the patch and copy the data to an NTFS volume and re-install the patch

- refrain from installing the patch entirely

The MS Support engineer even made a personal note:

In any case if you use NTFS I personally recommend using NTFS instead of ReFS, as ReFS is still immature as a file system as has several bugs.

I did let them know my frustration about this issue, since they broke it and still don't have a solution or even a cause after several weeks.

Wow that is quite the response. Passing the buck. Sheesh :rolling_eyes:

Page 5 / 5

Update #1:

Update #2:

Update #3:

Comment

Sign up

Login to the community

Scanning file for viruses.

This file cannot be downloaded