Data compression

Hi ​@Nikks -

As stated in the User Guide, Compression is recommended when using Dedup appliances:

https://helpcenter.veeam.com/docs/backup/vsphere/compression_deduplication.html?ver=120

When enabling Compression (is recommended most times, regardless), you can use a bit more compute resources on your Proxies. But, you shouldn’t notice to much latency there. Using Compression can actually help speed up your Job performance in most cases. You enable it in the Storage > Advanced button > Storage tab of your Jobs:

https://helpcenter.veeam.com/docs/backup/vsphere/backup_job_advanced_storage_vm.html?ver=120

Let me know if you have further questions.

Best.

And, as you shared you have an ‘offsite location’...in case you’re using Replication Jobs, the Compression config area for those Jobs is a bit different:

https://helpcenter.veeam.com/docs/backup/vsphere/replica_advanced_traffic_vm.html?ver=120

Make sure you have a Proxy and each end of your Rep Jobs → source and target.

Best.

Thanks for the answer. Right now our data domain (backup target) utilization is 95+%. so it is possible to go with high compression?

In addition to to what ​@coolsport00 shared, keep in mind that enabling compression will not affect previous backup files. Though it will take effect on all subsequent backup files without requiring an Active Full.

It sounds like you are asking about storing backups to an off-site Data Domain deduplication appliance. If the question is regarding leveraging Veeam compression, then first let me confirm your setup:

The Veeam gateway server configured for your Data Domain should be in the same offsite location.

Once this is done, you can enable high compression on the backup job. This will mean you have less data to transfer to the offsite location. However, compressed data is less likely to be deduplicated by the Data Domain so you should enable the setting to decompress data before storing them to the Data Domain appliance (Decompress backup data blocks before storing) - see https://helpcenter.veeam.com/docs/backup/hyperv/dsa_repository_repository.html?zoom_highlight=decompress+backup+data+blocks+before+storing&ver=120#configuring-advanced-repository-settings

If you do not have this setting enabled today, this could mean your deduplication ratio is much less than it should be.

Another question, do you have encryption enabled in your backup jobs?

yes, we have encryption enabled

Hi ​@Nikks,
I have to correct ​@coolsport00 here, as compression is not recommended for deduplication appliances. Actually it’s disabled on repository level, if you use the recommended settings: https://www.veeam.com/kb1745?ad=in-text-link#vendordocs

Having compression enabled on repository level would have an negative impact on the total deduplication level and therefore consume even more space. 

Also your existing backups on the Data Domain won’t be affected by changing the compression settings. The only way to lower the utilization would be to also lower the retention of your backups.

​@regnor is correct...I misread the Guide for the “dedup-freindly” configuration in the Compression settings. Oops! 😂 Thanks Max!

So ​@Nikks ...to get the best storage benefit for your DD appliance, you really want to utilize the best of the 2 storage features, dedup vs compression. In normal storage circumstances, you can use both to achieve the best ‘bang for your buck’ in storage utilization. You can use Compression with DD, but as ​@haslund suggests, your dedup ratio will be lower...thus, is not recommended.

So, if you’re just about at capacity as you state, I suggest disabling Compression to see where you’re utilization is at. You don’t need an Active Full for the change. Compression changes happen at next Job run without Fulls needed. The Limitations/Considerations area of the User Guide also suggests not having encryption enabled as that also has a negative effect on dedup ratio (4th bullet item in the link below):

https://helpcenter.veeam.com/docs/backup/vsphere/dell_dd.html?ver=120

Review the above and make the necessary adjustments and see how your storage utilization pans out. Regardless, you probably need to think about adding storage to your environment.

Best.

While in general, encryption on your backup jobs is heavily recommended, this is problematic when storing backups to a deduplication appliance as it means the data cannot be deduplicated (due to the encryption making all blocks unique).

Please consider if your data protection requirements need to be re-evaluated (maybe less retention?), disable encryption? or if perhaps the right thing is to keep your current setup and instead invest in more backup capacity.

Maybe it‘s an option for you, that only on the primary site encryption is enabled.

As ​@haslund stated, encryption enabled within a job to a DataDomain will nearly ‚kill‘ your DeDup ratio.
This is not the case, with ReFS or XFS in normal/hardened Repositories (but here we have normally a 3:1 - 4:1 ratio).

I’m quite sure, your utilization problem is related to your encryption setting, not on your compression level. 

…have also a look at: https://bp.veeam.com/vbr/3_Build_structures/B_Veeam_Components/B_backup_repositories/datadomain.html
 

On some of our clients with DataDomains we have around 20:1 ratios (or even more)… You can also check the Deduplication ratio on your DD via it‘s Web-UI or by CLI.

Edit (some hints/example from DELL):

• # filesys show compression
• # mtree show compression /data/col1/veeam-example

       Pre-Comp = Data written before compression
       Post-Comp = Storage used after compression
       Global-Comp Factor = Pre-Comp / (Size after de-dupe)
       Local-Comp Factor = (Size after de-dupe) / Post-Comp
       Total-Comp Factor = Pre-Comp / Post-Comp
       Reduction % = ((Pre-Comp - Post-Comp) / Pre-Comp) * 100

Ahi ahi ahi... never use data domain over than 85%. Now you've poor performance and deduplication has been reduced. 

Data Domains also handle encryption at rest on their own. You don’t need it in Veeam with DataDomain if you have DD encryption available. This is handled by the appliance itself, and there is no interaction between backup applications and data domains for this.

So your ideal model should be to use DD encryption always and disable it in the Veeam jobs. If you do further Backup Copies from the DD to non-DD storage, enable encryption on _those_ jobs. The data will be encrypted as soon as it is read from the source DD by our datamover.

Since you're using Data Domain as your backup target, it's generally recommended to let Data Domain handle deduplication and compression instead of enabling compression in Veeam. Enabling Veeam compression on top of Data Domain’s deduplication can lead to inefficiencies and increased storage usage.

Potential risks & impact:

• Increased CPU load on the backup server.
• Longer backup processing times, depending on the compression level.
• Reduced deduplication efficiency, as pre-compressed data is harder to deduplicate.

If storage utilization is a concern, consider adjusting retention policies, using synthetic full backups, or expanding storage capacity rather than enabling compression within Veeam. Hope this helps!