Analyse ReFS space savings with Block cloning

  • 20 October 2020
  • 6 comments
  • 2876 views

Userlevel 7
Badge +13

ReFS (Resilient File System) is a file system by Microsoft that provides - since Windows Server 2016 - a feature named Block cloning. Basically it uses pointers to already existing blocks instead of copying or re-creating them. This saves time and disk space. Veeam uses this feature for synthetic operations like merging incremental backups with full or synthetic full.

In my opinion Microsoft does not offer a proper utility to analyse ReFS space savings with Block cloning. I think there are at least three ways to show the space benefit of Block cloning:

Windows explorer

The native way to calculate saved space is to simply compare the sum of all files in a partition to the used space of the same partition. Because the partition view includes the space savings and file view does not, the difference shows saved space.

As you can see, there are about 44,4 GB space savings. Keep in mind, the properties of the partition includes the file system's overhead.

Application

Veeam shows capacity, free and used space in their repository view. The following screenshot shows a Veeam repository on the same partition as before (G:):

The outcome is near the same as before. Difference here is that Veeam just take files into account created by Veeam. This could be interesting when analyzing a volume including files from other applications.

blockstat.exe

First two options have one in common: they compare to the partition properties. This third option provides an insight on file level. The utility can be downloaded here. Timothy also provides some great PowerShell code snippets.

Here I will demonstrate the basic feature of this tool. Basically it analyses used blocks of a list of files. Blocks that are getting pointed to are summed up depending on the number of pointers. See the screenshot for clarification:

The multipliers beneath Sharing show how often these bytes are referred to. The counter includes the data itself as self-reference. Therefore this counter must be reduced by one for analyzing space reduction. See this excel-calculation according to the screenshot.

Actually shared values show space savings

blockstat.exe usage

For creating such a file level report, blockstat.exe needs just a list of files to analyse. This can be accomplished by a simple PowerShell command that lists necessary files and write them to a text file.

Get-ChildItem -Path g: -Recurse -Include @("*.vbk"; "*.vbr"; "*.vib") | ForEach-Object { $_.FullName } | Set-Content -encoding unicode "c:\refs_analysis\vbks.txt"

Because of Veeam, fulls and incremental files are listed. This command created the list of files, used in the screenshot above. To run blockstat.exe run this command:

C:\_install\script\blockstat.exe -i C:\refs_analysis\vbks.txt

It should be clear to replace these paths by your own.

Notes

  • The difference of saved space of the first two options compared to blockstat.exe in these examples can be explained by the file system meta data. blockstat.exe sums blocks that are use multiple times and do not take the used partition size into account.
  • When backing up VMware vSphere environments, using Changed Block Tracking, it is also very interesting how many block in what size are selected for backup. To analyse this, see my post What is the size of a changed block tracking block.


6 comments

Userlevel 7
Badge +17

Hi @vNote42 , I have played around with blockstat.

Nice tool. But it takes a lot of time for processing. I wanted to integrate this in a script which calculates the space savings for several repositories that are located on the same ReFS. Runs more than two hours and ends with errors.

I will try to give smaller chunks to blockstat and sum the results up. Perhaps this will take less time.

Userlevel 7
Badge +13

Hi @vNote42 , I have played around with blockstat.

Nice tool. But it takes a lot of time for processing. I wanted to integrate this in a script which calculates the space savings for several repositories that are located on the same ReFS. Runs more than two hours and ends with errors.

I will try to give smaller chunks to blockstat and sum the results up. Perhaps this will take less time.

You are right, Joe! I noticed in my testing too that it takes very long. 

Counterpart in Linux (xfs_bmap) is quite faster.

Userlevel 7
Badge +22

Hi Guys, I was just reading this as well. I have used the utility but as Joe says it takes a long time. My challenge, like many others here I believe who are service providers, is to figure out “per tenant” how much space they are really using. If the Tenant turns on GFS or weekly synthetic fulls then the issue crops up. So reporting will show for example 100TB being used while in reality there is only 20TB.

What I have noticed with GFS Monthly fulls is that initially with every new Month the added usage is minimal, but as time goes by and blocks are changing in the live filesystem the storage starts to shoot upwards. 

Badge

Has anyone seen anything new on reporting the space savings for XFS repo?

Userlevel 7
Badge +17

There was atool for XFS…. I do not remember the name at the moment, sorry. I will look for it tomorrow.

Userlevel 7
Badge +22

I thought I saw someone had scripted away to see that in XFS. I will dig as well.

Comment