• EN
Question

GFS by multiple jobs?

K
Userlevel 3

I studied about Veeam’s GFS, as I understand, it’ll consume a lot of disk space to work? 

 

If I want to keep 31 daily, 4 weekly, 12 monthly, I’ll end up with 1+4+12 = 17 full copies disk space + 30 daily incrementials ? 

 

What if I create 3 different jobs on the same VM, daily (30days), weekly (4 weeks), monthly (12 months)

I’ll end up with:

one daily full + 29 daily incrementals

one weekly full + 3 weekly incrementals

one monthly full + 11 monthly incrementals 

Total 3 full + 43 increments, should use much less space than build-in GFS?

 

 

 

 

 

8 comments

MicoolPaul
Userlevel 7
Badge +20

Hi,

 

Things like this:

one daily full + 29 daily incrementals

Indicate you’re using reverse incremental or incremental without periodic fulls. Otherwise if you’re doing incremental with synthetic or active fulls on say a weekly basis you’d have more fulls.

 

Also, GFS assigns “flags” to full backups to avoid a duplication of fulls.

 

Consider if you’ve got 4 weeks in a month, and you create a monthly full on the first week of the month.

On that first week, Veeam won’t create two fulls, one for a monthly and one for a weekly, it creates one full, it assigns it both weekly and monthly “flags” to it. When the weekly backup is no longer needed by retention, the weekly flag gets removed. Once all flags are removed it is purged. This works if you added yearly to the mix as well.

 

This is described here: https://helpcenter.veeam.com/docs/backup/vsphere/gfs_how_flags_assigned.html?ver=120#algorithm-for-multiple-flag-types

 

We can be more specific on what you should expect if you give us details on which incremental/reverse incremental backup types you’re using.

 

one weekly full + 3 weekly incrementals

one monthly full + 11 monthly incrementals 

 

Your issue with this is that your incrementals rely on the full, the whole point of GFS is to create independent restore points that don’t require any other files in the chain to be recoverable. If you lost your monthly full in this example, all incrementals over that year period are useless.

 

What are you backing up to? If you’ve got XFS or ReFS storage, then synthetic fulls will negate the need to consume the additional disk space of these fulls, and instead will use block cloning to reference the existing blocks that are identical to other full/incremental files in the backup history, much more efficient. And if you deleted a full/incremental, the file system is aware of the other files referencing those blocks and keeps those necessary blocks until no longer needed.

 

Hope this helps

Michael Paul - #VeeamLegend | #VeeamVanguard | Veeam Certified Architect | VMware vExpert | https://micoolpaul.com | Twitter: @MicoolPaul | Mastodon: @micoolpaul@masto.nu | Bluesky: @micoolpaul.bsky.social
K
Userlevel 3

Hi Michael,

 

The GFS assigns “flags” would help save me a few fulls but in my 30D/4W/12M case it’ll eventually still have 12 full monthly at least right ? 

 

I’m trying to backup ~10TB size of Windows SMB file share ,I wish to reach longest possible of RPO (31D/4W/12M/7Y) with least amount of disk space, and avoid having 365 forever incremental daily for performance and data integrity safety. My backup store is an iSCSI mount and I can do snapshot on LUN level so I’m not too worry about losting the monthly full, and I think just 11 monthly incremental should be safe and not too demaging for a backup-chain.

 

About ReFS, I don’t know much about it, and sure I’ll look into it, thanks. 

 

JMeixner
Userlevel 7
Badge +17

Yes, the weekly, monthly, quarterly and yearly backups are always full backups with a GFS job.

This is the sense of a GFS Jobs, to have independent restore points for the different retentions.

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
dloseke
Userlevel 7
Badge +6

I tend to skip weekly’s lately and run 30 daily’s and then move on to monthly’s and yearly’s.  But REFS and XFS are going to be your saviors here.  The other thing I do is keep my daily/weekly/monthly on site, and then keep longer monthly’s and yearlys at a remote site/in the cloud if needed.

Derek M. Loseke, Senior Systems Engineer | Veeam Legend 2022-2023 | VMSP/VMTSP | VCP6-DCV | VSP/VTSP | CCNA | https://technotesanddadjokes.com | @dloseke
K
Userlevel 3

@dloseke Yes I planned to do daily, monthly and yearly only. I’ve 30 daily so I don’t actually need weekly.

 

How does ReFS help here? I know some about btrfs , I think ReFS is similiar, some CoW magic ?

 

MicoolPaul
Userlevel 7
Badge +20

@dloseke Yes I planned to do daily, monthly and yearly only. I’ve 30 daily so I don’t actually need weekly.

 

How does ReFS help here? I know some about btrfs , I think ReFS is similiar, some CoW magic ?

 

As I said in my previous post, if you’re using ReFS or XFS it’ll use block cloning for efficiency.

 

Have a read of this:

https://helpcenter.veeam.com/docs/backup/vsphere/backup_repository_block_cloning.html?zoom_highlight=block+cloning&ver=120

 

And if you want a walkthrough of it with some nice & descriptive imagery, check out the legendary @jorge.delacruz’s blog post: https://jorgedelacruz.uk/2020/03/19/veeam-whats-new-in-veeam-backup-replication-v10-xfs-reflink-and-fast-clone-repositories-in-veeam/

 

The end result will be the same. Your “synthetic full” backups will re-use existing blocks from previous backups, but the key benefit of this over a long backup chain is that at the file system level, if a backup in the chain gets deleted that is needed by a synthetic full, the blocks referenced by it aren’t deleted, but instead the ownership  of the needed blocks are changed, and only blocks no longer needed by any file are purged. Effectively this makes your synthetic fulls consume similar space to an incremental. Imagine having weekly/monthly/yearly incrementals in GFS, that’s what fast clone gives you.

 

Microsoft also have a great, in-depth description of how block cloning works within ReFS: https://learn.microsoft.com/en-us/windows-server/storage/refs/block-cloning

Michael Paul - #VeeamLegend | #VeeamVanguard | Veeam Certified Architect | VMware vExpert | https://micoolpaul.com | Twitter: @MicoolPaul | Mastodon: @micoolpaul@masto.nu | Bluesky: @micoolpaul.bsky.social
keiranshelden
Userlevel 4
Badge +1

About ReFS, I don’t know much about it, and sure I’ll look into it, thanks. 

 

Some quick considerations around ReFS (While I love the technology and use it for all my Repos) The block cloning is only affective with Synthetic Fulls and incremental backups, where if you  had to restore a VM for any reason, the blocks will not longer be seen the same with CBT being reset, along with the vmID changing - so if you were to restore and then start backing up the restore VM, you will end up creating a fresh full and take up the equivilent amount of storage.   Just worth keeping in mind, if you end up in that situation. 

Otherwise, it is a great solution. - Make sure you read the Release notes before applying the Windows updates for it. :) 

vExpert ‘14-‘23 | NSX ‘18 | Drummer | @Techfieldday delegate | @tech4breakfast Australian Sr. Field Correspondent
K
Userlevel 3

@MicoolPaul thank you for your block cloning references, it really help! 

 

Comment


Terms & Conditions