Solved

Undestanding Backup Copy Jobs


Userlevel 3
Badge +1

Hi to all,

I try to understand the behavior of my lab jobs.

I have a backup jobs (of one VM) with the following settings:

Scheduled: every days at 22:00

Retention: 7 days

Synthetic Full: Saturday

GFS: 12M, 5Y

Repository: Hardened for 7 days

enabled a secondary copy to backup copy job with following settings:

Scheduled: every days at 08:00

Retention: 7 days

GFS: 12M, 5Y

Repository: Deduplicate storage

On backup jobs all work fine but on backup copy jobs appears only the first full (example of today) and then all incremental until the end of June and at 30 June has copied (created) a full backup (MY) not deleting any of the previous backups.

  • Restore point

 

  • Repository

 

icon

Best answer by vNote42 15 June 2022, 16:15

View original

20 comments

Userlevel 7
Badge +13

I would highly recommend to read this blog post by Hannes Kasparick:

https://www.veeam.com/blog/backup-copy-job-it-can-do-more-than-just-a-copy.html

With v11 retention policy has changed in backup copy jobs. It is now like the retention in primary job. What are your retention settings (also GFS) in copy job?

Userlevel 7
Badge +17

Are you ahead of time? You have backups until July 3rd 2022. I am at June 15th 2022 only….

Perhaps this is the problem. All backups before June 15th seem to be deleted...

Userlevel 3
Badge +1

I would highly recommend to read this blog post by Hannes Kasparick:

https://www.veeam.com/blog/backup-copy-job-it-can-do-more-than-just-a-copy.html

With v11 retention policy has changed in backup copy jobs. It is now like the retention in primary job. What are your retention settings (also GFS) in copy job?

In all jobs the retention are the same:

Retention 7 days

GFS 12M, 5Y

Userlevel 3
Badge +1

Are you ahead of time? You have backups until July 3rd 2022. I am at June 15th 2022 only….

Perhaps this is the problem. All backups before June 15th seem to be deleted...

Hi this is a lab to understand the behavior of this simulation. I have changed the data/time manually on all system.

Userlevel 7
Badge +17

OK, I understand 😎

Userlevel 3
Badge +1

I would highly recommend to read this blog post by Hannes Kasparick:

https://www.veeam.com/blog/backup-copy-job-it-can-do-more-than-just-a-copy.html

With v11 retention policy has changed in backup copy jobs. It is now like the retention in primary job. What are your retention settings (also GFS) in copy job?

In all jobs the retention are the same:

Retention 7 days

GFS 12M, 5Y

@vNote42 I have read better the blog post and it clearly describes that the backup copy job works as “forever incremental”.

It is not clear to me only why it did not delete the backup points from June 14 to June 29

Userlevel 7
Badge +13

I guess the problem is, you do not have weekly GFS restore points set. In the screenshot you provided, VBR is not able to delete that much restore points that 7 remains. On 15.6. you have a full because you started the job then. On 1.7. you have a full because this a monthly backup. I am quite sure, if you run the job another 6 days, VBR will delete increments between 15.6. and 1.7. … If no faulty reasoning 😁

Userlevel 7
Badge +6

Because you are using 7 days of retention and Forward Incremental backups, the incrementals older than 26/06/2022 cannot be deleted because they are part of the chain going back to the full on 14/06/2022 and are reliant on that full restore point.  If you had weekly restore points in your GFS retention policy, then you would have incrementals back to 26/06/2022 I believe.  If you run 4 more days to 06/07/2022 (July 6th - sorry, I’m American so I have to clarify that I’m using the D/M/Y format rather than our typical M/D/Y), then it should drop everything beyond 30/06/2022 *I think*.  I say I think because you also have yearly retention, but I’m pretty sure that part of the chain would no longer be required since you’re keeping 7 days and the data older than the your Monthly/Yearly restore point wouldn’t be usable for the Monthly/Yearly GFS restore points.  I could be wrong on that part though.  Basically, I’d run your simulation forward another week and I think you’ll see those older incrementals and Full drop off. 

I will note that if you were running Reverse Incrementals, the newest restore point is always the full and I would expect those old incrementals to have dropped off as they won’t rely on an older full restore point.  However, Reverse Incrementals seem to have fallen out of favor as their are a bit more IO intensive and I don’t see much of an advantage to them in most cases now that I’ve wrapped my head around how the retention works.

Userlevel 7
Badge +17

Yeah, the cause seem to be the GFS fulls. Checked my copy jobs (without GFS) and they are using reverse incremental with the full backup the newest restore point.

Let's wait some days until the new chain is complete with 7 days and see what happens then.

Userlevel 7
Badge +13

Because you are using 7 days of retention and Forward Incremental backups, the incrementals older than 26/06/2022 cannot be deleted because they are part of the chain going back to the full on 14/06/2022 and are reliant on that full restore point.  If you had weekly restore points in your GFS retention policy, then you would have incrementals back to 26/06/2022 I believe.  If you run 4 more days to 06/07/2022 (July 6th - sorry, I’m American so I have to clarify that I’m using the D/M/Y format rather than our typical M/D/Y), then it should drop everything beyond 30/06/2022 *I think*.  I say I think because you also have yearly retention, but I’m pretty sure that part of the chain would no longer be required since you’re keeping 7 days and the data older than the your Monthly/Yearly restore point wouldn’t be usable for the Monthly/Yearly GFS restore points.  I could be wrong on that part though.  Basically, I’d run your simulation forward another week and I think you’ll see those older incrementals and Full drop off. 

I will note that if you were running Reverse Incrementals, the newest restore point is always the full and I would expect those old incrementals to have dropped off as they won’t rely on an older full restore point.  However, Reverse Incrementals seem to have fallen out of favor as their are a bit more IO intensive and I don’t see much of an advantage to them in most cases now that I’ve wrapped my head around how the retention works.

Copy job is always forward incremental. Doesn’t matter what type (forward/reverse) primary job is.

Userlevel 7
Badge +6

Copy job is always forward incremental. Doesn’t matter what type (forward/reverse) primary job is.

 

Ah yes...thank you.  Forgot about that detail on the copy jobs.

Userlevel 7
Badge +6

Yeah, the cause seem to be the GFS fulls. Checked my copy jobs (without GFS) and they are using reverse incremental with the full backup the newest restore point.

Let's wait some days until the new chain is complete with 7 days and see what happens then.

 

Yes, I’d love to see what @fspadaro  can show us after we get a few more days mocked up.  Please report back, if not just for our own curiosities sake, and to validate our theories.  :-)

Userlevel 3
Badge +1

Hi to all, 

unfortunately I have to delete all test for less resource but I have replicated the lab starting on 15 june and following the result:

Job start on 15 june with restore point 7days (synthetic Saturday) and GFS 12M 5Y

  • Screenshot on 25 june at 10:05 – Repository Immutable and Backup Copy Job
  • Screenshot on 26 june at 10:05 – Repository Immutable and Backup Copy Job

________________________________________________________________________________________

  • Screenshot on 02 july at 10:05 – Repository Immutable and Backup Copy Job
  • Screenshot on 03 july at 10:05 – Repository Immutable and Backup Copy Job
  • Screenshot on 03 july at 10:05 – Restore Point Immutable and Backup Copy Job

 

_______________________________________________________________________________________________

  • Screenshot on 09 july at 10:05 – Repository Immutable and Backup Copy Job
  • Screenshot on 10 july at 10:05 – Repository Immutable and Backup Copy Job
  • Screenshot on 10 july at 10:05 – Restore Point Immutable and Backup Copy Job

 

Userlevel 7
Badge +13

Hi to all, 

unfortunately I have to delete all test for less resource but I have replicated the lab starting on 15 june and following the result:

Job start on 15 june with restore point 7days (synthetic Saturday) and GFS 12M 5Y

Ma hai modificato manualmente l’orologio del sistema per avere quelle date?

Edit. Engl.
But have you manually edited clock on Windows to get those dates?

Userlevel 7
Badge +13

@fspadaro two things, in the first post you write:

Retention: 7 days
Repository: Hardened for 7 days

but I remember that using the same retention for backups and immutable backups could generate issues…

And, is that a real backup copy job? It seems just another backup job with that logs… but maybe I’m wrong.

Userlevel 3
Badge +1

Hi to all, 

unfortunately I have to delete all test for less resource but I have replicated the lab starting on 15 june and following the result:

Job start on 15 june with restore point 7days (synthetic Saturday) and GFS 12M 5Y

Ma hai modificato manualmente l’orologio del sistema per avere quelle date?

Edit. Engl.
But have you manually edited clock on Windows to get those dates?

Si certamente

Userlevel 3
Badge +1

@fspadaro two things, in the first post you write:

Retention: 7 days
Repository: Hardened for 7 days

but I remember that using the same retention for backups and immutable backups could generate issues…

And, is that a real backup copy job? It seems just another backup job with that logs… but maybe I’m wrong.

This is a lab to understand backup copy job from an hardened repo and I cannot wait a lot of days of immutable; say that I think there isn’t a know issue to use same retention and same hardened retention, have you same documentation about this?
Where does your perplexity come that is not a BCJ? 

Userlevel 7
Badge +13

@fspadaro I can’t find the documentation I read, but same way is suggested by Veeam here

https://forums.veeam.com/object-storage-f52/help-sizing-up-immutable-storage-t79850.html

Userlevel 7
Badge +13

@fspadaro Does the copy job now work as you expect it to? 

Userlevel 3
Badge +1

@fspadaro Does the copy job now work as you expect it to? 

Yes it seems to work fine….now I need to check after another month.

Thanks at all

Comment