I work pretty heavily with backups in my current role, and we use your product and another product with some of our clients. The other product has a feature that is incredibly helpful and I think would be an amazing feature to add to Veeam. How feasible would it be to add a “Force Retention” feature?
I really believe we need this for situations when we run into low disk space. In Veeam’s current state, if I run into an issue where I need to reclaim disk space, I am forced to break my backup chain and delete data. If I could lower retention on my backup and then force retention, that would effectively eliminate the need for me to have to lose data and break a backup chain. For example: if I am in a situation where I am getting failures due to low disk space, and I have retention on said backup job set to 30 days, if I could cut retention down to, say, 20 days, and force retention, I could maintain my current backup chain and then address the disk space issue accordingly (adding more disk space or some other solution). If I have to run a backup to get retention to occur, I am crap out of luck because I don’t have enough disk space to run said backup and I am forced to delete data and break my backup chain.
This feature would be amazing and I would genuinely appreciate it if you would consider it and potentially add it. It is a needed feature and would be incredibly helpful.
Thank you for the great product and hard work!
Page 1 / 1
I work pretty heavily with backups in my current role, and we use your product and another product with some of our clients. The other product has a feature that is incredibly helpful and I think would be an amazing feature to add to Veeam. How feasible would it be to add a “Force Retention” feature?
I really believe we need this for situations when we run into low disk space. In Veeam’s current state, if I run into an issue where I need to reclaim disk space, I am forced to break my backup chain and delete data. If I could lower retention on my backup and then force retention, that would effectively eliminate the need for me to have to lose data and break a backup chain. For example: if I am in a situation where I am getting failures due to low disk space, and I have retention on said backup job set to 30 days, if I could cut retention down to, say, 20 days, and force retention, I could maintain my current backup chain and then address the disk space issue accordingly (adding more disk space or some other solution). If I have to run a backup to get retention to occur, I am crap out of luck because I don’t have enough disk space to run said backup and I am forced to delete data and break my backup chain.
This feature would be amazing and I would genuinely appreciate it if you would consider it and potentially add it. It is a needed feature and would be incredibly helpful.
Thank you for the great product and hard work!
What chain type are you using? Reverse Incremental or Forward Incremental? I could see this being a benefit when using reverse incremental to purge the older points automatically but it wouldn’t be as effective with forward incremental as you’d either need to transform your forward incrementals for a chain into a reverse chain on another full (which if you’re low on disk space could be problematic) or you’d need to purge an entire chain anyway. Good shout all the same :)
I work pretty heavily with backups in my current role, and we use your product and another product with some of our clients. The other product has a feature that is incredibly helpful and I think would be an amazing feature to add to Veeam. How feasible would it be to add a “Force Retention” feature?
I really believe we need this for situations when we run into low disk space. In Veeam’s current state, if I run into an issue where I need to reclaim disk space, I am forced to break my backup chain and delete data. If I could lower retention on my backup and then force retention, that would effectively eliminate the need for me to have to lose data and break a backup chain. For example: if I am in a situation where I am getting failures due to low disk space, and I have retention on said backup job set to 30 days, if I could cut retention down to, say, 20 days, and force retention, I could maintain my current backup chain and then address the disk space issue accordingly (adding more disk space or some other solution). If I have to run a backup to get retention to occur, I am crap out of luck because I don’t have enough disk space to run said backup and I am forced to delete data and break my backup chain.
This feature would be amazing and I would genuinely appreciate it if you would consider it and potentially add it. It is a needed feature and would be incredibly helpful.
Thank you for the great product and hard work!
What chain type are you using? Reverse Incremental or Forward Incremental? I could see this being a benefit when using reverse incremental to purge the older points automatically but it wouldn’t be as effective with forward incremental as you’d either need to transform your forward incrementals for a chain into a reverse chain on another full (which if you’re low on disk space could be problematic) or you’d need to purge an entire chain anyway. Good shout all the same :)
It depends on the client. I use forever forward on some of them, weekly synthetics on some, and periodic fulls on others.
I am intentionally not naming the product, but with this other product, if I ever run into a disk space issue, I can go to settings, hit force retention, and retention happens then and there without the need of a backup. With whatever backup type I am using, with Veeam, I have to run a backup in these situations, and I am screwed at that point because a backup won’t run due to their being no disk space, and Veeam’s current solution is for me to delete data and break my chain. I do not like doing that and if I had the ability to just lower retention some and then tell Veeam to get rid of those restore points without the need of running a backup, that would eliminate that issue for me and buy me time to address the disk space issue. I hope that makes sense and answers the question.
I am still new to backup stuff and am actively learning :)
Thanks for sharing. Veeam’s angle has always been that your retention policy is the minimum amount of restore points to keep, as defined by you, and it’s recommended to set a suitable % alert threshold for when you need to address this as the repository should always have sufficient working space for the backups.
Are you using these volumes with something other than Veeam Backup jobs? As if you’re using reverse incremental for example, your chain will automatically truncate after each backup to maintain your retention point. It may be worth looking at a different chain type if it’s more suitable.
Mhh, this would be a useful function, but I think it is very difficult to realize.
When you are doing forward - for example -incremental backups with a retention of 4, then you have up to 7 versions in your repository before an old version is deleted.
Theoretically the version from sunday could be deleted on thursday. But the backups from monday to wednesday depend on the full backup from sunday - so it cannot be deleted without invalidating the depending backups and violating the retention of 4. After the backup is done on the next sunday and a new backup chain with 4 versions is complete the complete chain from sunday to wednesday is deleted.
To be able to delete a version earlier you would have to migrate one of the incremental into a full or you would habe to change the whole thing into a reverse incremental chain. Both approaches need space in your reporsitory.
I could imagine a function you are describing with a file based backup system like Spectrum Protect. There you can delete a number of versions of each file without breaking depending backups. But even there a retention change is done with the next backup - when I remember correct.
ok, Michael was faster to respond. I didn’t refresh the page after being “disturbed” by a customer while writing my answer… Sorry.
Thanks for sharing. Veeam’s angle has always been that your retention policy is the minimum amount of restore points to keep, as defined by you, and it’s recommended to set a suitable % alert threshold for when you need to address this as the repository should always have sufficient working space for the backups.
Are you using these volumes with something other than Veeam Backup jobs? As if you’re using reverse incremental for example, your chain will automatically truncate after each backup to maintain your retention point. It may be worth looking at a different chain type if it’s more suitable.
I’m glad you mentioned that. I am still trying to understand all of the chain types and reverse incremental is one that I am not super familiar with.
When you’re in a situation where you have limited disk space and not necessarily a way to get more, would reverse incremental be the way to go?
Thank you so much for the replies.
Mhh, this would be a useful function, but I think it is very difficult to realize.
When you are doing forward - for example -incremental backups with a retention of 4, then you have up to 7 versions in your repository before an old version is deleted.
Theoretically the version from sunday could be deleted on thursday. But the backups from monday to wednesday depend on the full backup from sunday - so it cannot be deleted without invalidating the depending backups and violating the retention of 4. After the backup is done on the next sunday and a new backup chain with 4 versions is complete the complete chain from sunday to wednesday is deleted.
To be able to delete a version earlier you would have to migrate one of the incremental into a full or you would habe to change the whole thing into a reverse incremental chain. Both approaches need space in your reporsitory.
I could imagine a function you are describing with a file based backup system like Spectrum Protect. There you can delete a number of versions of each file without breaking depending backups. But even there a retention change is done with the next backup - when I remember correct.
In the scenario you are describing, why do 7 restore points sit in the repository rather than just 4 and the oldest 5th be deleted after each backup? This question is due to ignorance on my part. I apologize for my lack of understanding on this. Backup theory still just boggles my mind at this point and I find it difficult to understand.
Thank you so much for your responses and patience. I genuinely appreciate.
Would it be helpful to mention the other product we use?
Hello @bp4JC no problem, my beginnings with backup were hard, too
VEEAM uses with forward incremantal method backup chains with incremental backups which depends on the last full backup, which is here the odest in the chain. In the picture the backups from sunday to wednesday are one chain.
On thursday a new full backup (a synthetic one) is created. This is the fifth restore point in your repository and you could (in theory) delete the point from sunday and you would fulfill your retention of 4 versions.
But you cannot delete the restore point from sunday because the incremental backups from monday, tuesday and wednesday depend on the full backup from sunday. Without the full backup the incrementals are useless. In this case you would have only one restore point available (the restore point from thursday).
In order to fullfll your retention of 4 you have to wait until the next sunday, then you have a complete chain (Full from thursday and 3 incrementals depending on it). Now the old chain is deleted and all 4 restore points from the first sunday to wednesday are gone.
The “problem” are the depending incrementals. These contain the changes to the full backup only. When you want to restore the restore point from wednesday, then the full backup from sunday is used and the incrementals from monday to wednesday to add the changes to the full backup. Without the full backup the incrementals cannot be restored.
To avoid this you can use a reverse incremental chain. Here is the last restore point a (synthetic) full backup at every time.
With this method the full backup is the newest restore point and for older restore points one or more incrementals are used for restore. Now you can delete a number of the incrementals without loosing the ability to restore the latest versions…
I hope this explanation is more clear… please feel free to ask further, I will try to explain
Hello @bp4JC no problem, my beginnings with backup were hard, too
VEEAM uses with forward incremantal method backup chains with incremental backups which depends on the last full backup, which is here the odest in the chain. In the picture the backups from sunday to wednesday are one chain.
On thursday a new full backup (a synthetic one) is created. This is the fifth restore point in your repository and you could (in theory) delete the point from sunday and you would fulfill your retention of 4 versions.
But you cannot delete the restore point from sunday because the incremental backups from monday, tuesday and wednesday depend on the full backup from sunday. Without the full backup the incrementals are useless. In this case you would have only one restore point available (the restore point from thursday).
In order to fullfll your retention of 4 you have to wait until the next sunday, then you have a complete chain (Full from thursday and 3 incrementals depending on it). Now the old chain is deleted and all 4 restore points from the first sunday to wednesday are gone.
The “problem” are the depending incrementals. These contain the changes to the full backup only. When you want to restore the restore point from wednesday, then the full backup from sunday is used and the incrementals from monday to wednesday to add the changes to the full backup. Without the full backup the incrementals cannot be restored.
To avoid this you can use a reverse incremental chain. Here is the last restore point a (synthetic) full backup at every time.
With this method the full backup is the newest restore point and for older restore points one or more incrementals are used for restore. Now you can delete a number of the incrementals without loosing the ability to restore the latest versions…
I hope this explanation is more clear… please feel free to ask further, I will try to explain
Essentially, the 4 restore points (incrementals the one full) are all dependent on each other and even with the new full backup, The older incrementals are still dependent on the previous full backup and that stuff won’t be able to be deleted until the next full backup runs the following Sunday and a new complete chain is there. Because the incrementals are dependent on the previous full backup they are referencing, we can’t lose anything until we have another four viable restore points (including the full), and therefore the oldest can’t be deleted until we have a new full backup and three corresponding incrementals. Only then can we lose the old restore points because we now have a new complete set of 4. I hope I am making sense with this.
I think I understand...it’s just a mind bender for me. I am not sure why I struggle to understand it.
Let me ask you this. In our scenario, once we have the new full, wouldn’t it contain all of the data from the previous 3 incrementals and previous full? If that is the case, why couldn’t we just go ahead a lose the old 4 once we have the new full? I understand that the previous incrementals are dependent on the oldest full, but if we have a new full backup containing all of that data, couldn’t we just recover from the newest full if we needed to?
Hello @bp4JC no problem, my beginnings with backup were hard, too
VEEAM uses with forward incremantal method backup chains with incremental backups which depends on the last full backup, which is here the odest in the chain. In the picture the backups from sunday to wednesday are one chain.
On thursday a new full backup (a synthetic one) is created. This is the fifth restore point in your repository and you could (in theory) delete the point from sunday and you would fulfill your retention of 4 versions.
But you cannot delete the restore point from sunday because the incremental backups from monday, tuesday and wednesday depend on the full backup from sunday. Without the full backup the incrementals are useless. In this case you would have only one restore point available (the restore point from thursday).
In order to fullfll your retention of 4 you have to wait until the next sunday, then you have a complete chain (Full from thursday and 3 incrementals depending on it). Now the old chain is deleted and all 4 restore points from the first sunday to wednesday are gone.
The “problem” are the depending incrementals. These contain the changes to the full backup only. When you want to restore the restore point from wednesday, then the full backup from sunday is used and the incrementals from monday to wednesday to add the changes to the full backup. Without the full backup the incrementals cannot be restored.
To avoid this you can use a reverse incremental chain. Here is the last restore point a (synthetic) full backup at every time.
With this method the full backup is the newest restore point and for older restore points one or more incrementals are used for restore. Now you can delete a number of the incrementals without loosing the ability to restore the latest versions…
I hope this explanation is more clear… please feel free to ask further, I will try to explain
Essentially, the 4 restore points (incrementals the one full) are all dependent on each other and even with the new full backup, The older incrementals are still dependent on the previous full backup and that stuff won’t be able to be deleted until the next full backup runs the following Sunday and a new complete chain is there. Because the incrementals are dependent on the previous full backup they are referencing, we can’t lose anything until we have another four viable restore points (including the full), and therefore the oldest can’t be deleted until we have a new full backup and three corresponding incrementals. Only then can we lose the old restore points because we now have a new complete set of 4. I hope I am making sense with this.
I think I understand...it’s just a mind bender for me. I am not sure why I struggle to understand it.
Let me ask you this. In our scenario, once we have the new full, wouldn’t it contain all of the data from the previous 3 incrementals and previous full? If that is the case, why couldn’t we just go ahead a lose the old 4 once we have the new full? I understand that the previous incrementals are dependent on the oldest full, but if we have a new full backup containing all of that data, couldn’t we just recover from the newest full if we needed to?
There’s no guarantee that all the old data is there. We’re not just tracking “new” data added, but also changed data. Focusing on reverse incremental for a moment. When you take a new backup, Veeam injects the new data into your full, then ejects the data that is no longer valid in your latest backup (data modified or deleted) into a reverse incremental file to allow tracking of these differences. Retention enables you to track these changes, if you just want a latest copy only you could create an extremely low retention point reverse incremental job.
Let me ask you this. In our scenario, once we have the new full, wouldn’t it contain all of the data from the previous 3 incrementals and previous full? If that is the case, why couldn’t we just go ahead a lose the old 4 once we have the new full? I understand that the previous incrementals are dependent on the oldest full, but if we have a new full backup containing all of that data, couldn’t we just recover from the newest full if we needed to?
Yes, you understood this correct.
To your question:
If you delete all older restore points after the next full, then you are not able to restore to all single restore points before.
In most cases the requirement is to be able to restore to the state of each day, hour, whatever your client’s requirements are. Each restore point represents a “snapshot in time” and you want to recreate it if neccessary.
… just skimmed the answers to you feature request…
I support your request! Just last week this feature would have helped me to continue with backups rather than deleting to much restore points. I agree with @JMeixner that it is not that easy. When you are already at low diskspace, transforming backup could crash. This should be a part of the feature: make some calculations if it is even possible to to the enforcement.
I personally, did some time to internalize the difference between “forever forward incremental” and “forward incremental”. With forever, full is changed with every backup-run (oldest increment is merged into full). With forward (with periodic fulls), no file is changed after creation. When deleting restore points, a whole chain is deleted - as set out above by joe.
Great conversation here!
@bp4JC, you can also enter that as a feature request in the Veeam R&D Forums as we don’t ensure all of the FR’s are seen by Veeam Product Management in this community site.
Great conversation here!
@bp4JC, you can also enter that as a feature request in the Veeam R&D Forums as we don’t ensure all of the FR’s are seen by Veeam Product Management in this community site.
Thank you for the heads up on this. I have posted it there as well :)
I believe I know which product you are talking about and to be honest, when I started using Veeam, I had always asked for the feature that you requested. Even the concept of the number of recovery points against the number of days of backups was also new at that time(introduced recently in Veeam as well). So having worked on Backup tech for the last 15 years and over 5 different backup products, in my opinion, managing the backup disk space can be improved and there is certainly room for improvement.
Thanks for bringing it up.
This could really be useful. If you're low or out of disk space then it's always hard to free up disk space without destroying a part of the backup chain and loosing restore points. So +1
ok, Michael was faster to respond. I didn’t refresh the page after being “disturbed” by a customer while writing my answer… Sorry.
