• EN
Solved

Tape job issues if job can't complete in time

M
Userlevel 4

So I have an issue. We backup to disk (a Data Domain, in our case), and from there, copy to tape. So I have 2 jobs - a backup job that writes to the DD, and a tape job (set as secondary target). Backup job is set to make an active full once a week (usually on a Friday), and the other days are incrementals. Pretty standard so far.

My issue: on the weekend, we ran out of blank tapes in the library. I had to cancel the running tape job (which was trying to write out the Friday FULL backup). I got some new blanks, and loaded them. But the Monday tape job went to write out the (missed) Friday FULL backup, instead of just doing the Monday incremental. 

The FULL job is like 4 TB, no way it will finish in time before the next night’s incremental backup will start up. I had to change the scheduled start to “Not scheduled”, just so it would have enough time to finish (else it would have been interrupted by that evening’s incremental job). I don’t want to hold the source backup job - I want it to write out the incremental to disk for that night, as it should, I don’t want it to miss it’s scheduled start time. I want the tape backup to not go back and re-try the missed tape writing.

So how can I set the tape job to not do this - i.e., not go back and attempt to write out a previous missed backup? I only want it to do whatever that day’s scheduled backup job is (incremental). I don’t see any way to do that.

I hope I explained that clearly enough. Thanks for any insight.

icon

Best answer by Chris.Childerhose 14 December 2023, 16:53

View original

32 comments

JMeixner
Userlevel 7
Badge +17

Please try it and tell us the results. 😎👍🏼

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
M
Userlevel 4

As I say, we’ll see. What I want is for the job (total 4TB) to be written to multiple drives at once. Not 1 VM to 1 drive, the other VM to another drive, so that almost all 4TB is only written by 1 drive if 1 VM is larger than another. 

I have to use what I have ….

Chris.Childerhose
Userlevel 7
Badge +20

These are LTO-7 drives, although I had been using LTO-6 tapes (I finally have an order for LTO-7 tapes coming, now that we’ve utilized our stock LTO-6 tapes). And support does say “If you had multiple jobs or multiple tasks/VMs within a single job, then you would be able to utilize parallel processing the way you are trying to use it”, which seems to disagree with what you say. I take that statement to indicate that the important thing is not the amount of data, but the number of VMs in the job. Now, maybe it means that each VM will go to a different tape drive, I dunno, we’ll find out. Still waiting for the shipment of new tapes to come in ….

That is what the statement you got means the more VMs in a job the more tape drives it will use regardless of data size.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
JMeixner
Userlevel 7
Badge +17

In my environments each VM can be handled by another drive, but not backup files from one VM.

It is possible that there is a change in behavior with V12.1, I am on V12.0 with my tape environments at the moment….

And ok, LTO-7 has a transfer speed of 300MB/sec, but LTO-9 is still 50% faster 😎

 

BTW, I would interpret the statement from support the same way I said it.

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
M
Userlevel 4

These are LTO-7 drives, although I had been using LTO-6 tapes (I finally have an order for LTO-7 tapes coming, now that we’ve utilized our stock LTO-6 tapes). And support does say “If you had multiple jobs or multiple tasks/VMs within a single job, then you would be able to utilize parallel processing the way you are trying to use it”, which seems to disagree with what you say. I take that statement to indicate that the important thing is not the amount of data, but the number of VMs in the job. Now, maybe it means that each VM will go to a different tape drive, I dunno, we’ll find out. Still waiting for the shipment of new tapes to come in ….

JMeixner
Userlevel 7
Badge +17

This is the reason why I asked if you have per VM backup files activated 😎

Ok, when you have one VM in this job only, then one tape drive will be utilized.

And to include a second, unrelated VM into this job will not help, because the amount of data for your actual VM will not decrease and this will be handled by one tape drive even if you add several VMs.

The only solution I see for this is to use a newer and more powerful tape drive. Fir example a LTO-6 drive has atransfer speed of 150MB/sec, a LTO-9 drive has a transfer speed of 400MB/sec.

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
M
Userlevel 4

UPDATE:

I heard back from Support. Apparently, if your backup job has only 1 VM (as mine does), then Veeam will not utilize parallel processing (i.e., write to multiple tape drives at once). So I might change my job to backup a 2nd, un-related VM, just so that when the job writes to tape, it will utilize multiple tape drives at once.

The reason for this is because you are currently backing up a single backup job with a single VM in it (at least from what I can tell).  In VBR, one VM will not be written to multiple drives asynchronously. If you had multiple jobs or multiple tasks/VMs within a single job, then you would be able to utilize parallel processing the way you are trying to use it, but sadly, that doesn’t seem to be your current setup.
Chris.Childerhose
Userlevel 7
Badge +20

I have opened a ticket with support, to ask why I can’t utilize multiple drives for a tape job. We’ll see what they can figure out.

Probably the best route forward at this point and maybe they have some tweaks that can be done too.  Best of luck and let us know how it goes.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
M
Userlevel 4

I have opened a ticket with support, to ask why I can’t utilize multiple drives for a tape job. We’ll see what they can figure out.

Chris.Childerhose
Userlevel 7
Badge +20

Right-click the job name not the VM inside the job.  The new backup chain pertains to the entire job.

Nope.

 

 

Ok then you are already using the new backup chain format.  No need to worry about it.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
M
Userlevel 4

Right-click the job name not the VM inside the job.  The new backup chain pertains to the entire job.

Nope.

 

 

Chris.Childerhose
Userlevel 7
Badge +20

Did you upgrade your backup chains and enable the per-VM backup files?

If there are two files to write to tape only, then two drives can be used in parallel.

 

I don’t understand what you mean “upgrade your backup chains and enable the per-VM backup files”?

 

If you go under the Disk section in the console and then under your backup job you can click on the job and the Upgrade Backup Chain will be available.  That is what Joe means.

 

Don’t see that at all. Not under the job, or if right click and choose properties of the job.

 

I am running VBR 12.0.0.1420 P20230718. This was upgraded from an earlier v11.

 

 

Right-click the job name not the VM inside the job.  The new backup chain pertains to the entire job.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
M
Userlevel 4

I did check off the box to use all 3 drives on 1 job, that does have a 4TB FULL backup and a couple incrementals waiting. I then started the job. Only 2 tape drives are loaded, and only 1 is writing (the 3rd drive is unloaded and idle). So that option isn’t helping me, at least not yet ...

Give it some time to see if the third kicks in.  It tends to be slow when starting.  😋

 

Been over an hour, and still only 1 drive writing. Slow starting is one thing, but … LOL 🤐

M
Userlevel 4

Did you upgrade your backup chains and enable the per-VM backup files?

If there are two files to write to tape only, then two drives can be used in parallel.

 

I don’t understand what you mean “upgrade your backup chains and enable the per-VM backup files”?

 

If you go under the Disk section in the console and then under your backup job you can click on the job and the Upgrade Backup Chain will be available.  That is what Joe means.

 

Don’t see that at all. Not under the job, or if right click and choose properties of the job.

 

I am running VBR 12.0.0.1420 P20230718. This was upgraded from an earlier v11.

 

 

Chris.Childerhose
Userlevel 7
Badge +20

With VBR V12 was a new format of backup chains introduced.
Up to V11 all VMs in a job were put into one big backup file, V12 and up can put each VM in job into a own backup file.

 

Please find more information here:

https://helpcenter.veeam.com/docs/backup/vsphere/per_vm_backup_files.html?ver=120

This will help with processing to tape as well - FYI.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
JMeixner
Userlevel 7
Badge +17

With VBR V12 was a new format of backup chains introduced.
Up to V11 all VMs in a job were put into one big backup file, V12 and up can put each VM in job into a own backup file.

 

Please find more information here:

https://helpcenter.veeam.com/docs/backup/vsphere/per_vm_backup_files.html?ver=120

 

With this you will have smaller backup files for each VM which can be moved to tape individually and more tape drives can be utilized in parallel.

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
Chris.Childerhose
Userlevel 7
Badge +20

Did you upgrade your backup chains and enable the per-VM backup files?

If there are two files to write to tape only, then two drives can be used in parallel.

 

I don’t understand what you mean “upgrade your backup chains and enable the per-VM backup files”?

 

If you go under the Disk section in the console and then under your backup job you can click on the job and the Upgrade Backup Chain will be available.  That is what Joe means.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
M
Userlevel 4

Did you upgrade your backup chains and enable the per-VM backup files?

If there are two files to write to tape only, then two drives can be used in parallel.

 

I don’t understand what you mean “upgrade your backup chains and enable the per-VM backup files”?

 

Chris.Childerhose
Userlevel 7
Badge +20

I did check off the box to use all 3 drives on 1 job, that does have a 4TB FULL backup and a couple incrementals waiting. I then started the job. Only 2 tape drives are loaded, and only 1 is writing (the 3rd drive is unloaded and idle). So that option isn’t helping me, at least not yet ...

Give it some time to see if the third kicks in.  It tends to be slow when starting.  😋

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
JMeixner
Userlevel 7
Badge +17

Did you upgrade your backup chains and enable the per-VM backup files?

If there are two files to write to tape only, then two drives can be used in parallel.

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
M
Userlevel 4

I did check off the box to use all 3 drives on 1 job, that does have a 4TB FULL backup and a couple incrementals waiting. I then started the job. Only 2 tape drives are loaded, and only 1 is writing (the 3rd drive is unloaded and idle). So that option isn’t helping me, at least not yet ...

M
Userlevel 4

I have 3 tape drives. And one media pool, which is set for parallel processing to use up to 3 drives. The default for the tape jobs looks to be set to 2 (it’s unchecked, but the value shows 2). I could set it to 3.

Even so, not being able to skip sending a specific full to tape complicates things. I may open a ticket and ask support if there’s a way. Otherwise, if I miss one weekend, it will NEVER have enough time to catch up. And that can’t be right, not for an enterprise level program …

Thanks

Tick off the box and set the job to use all 3 drives.  Hopefully that helps get things over faster to complete for you.

 

Thanks, I’ll try that. Looks like the order for new tapes won’t come be delivered in time for this weekend, so I will probably have the same issue again …. 

I am also considering enabling jumbo Ethernet frames for the Veeam server and proxies (it’s already enabled for the DD). I dunno if it will help, but I suppose it can’t hurt …

 

Chris.Childerhose
Userlevel 7
Badge +20

Veeam has to make sure that the complete backup chain is written to tape. You cannot restore from the incrementals when then full from last Friday is missing.

So, you have to let the job finish. And yes, this can cause a missed incremental or an incremental at another tome of the day.

Do you have one tape drive only? If you have more, you can try to configure the tape job to use multiple drive in parallel to get the data faster to that tapes.

I have 3 tape drives. And one media pool, which is set for parallel processing to use up to 3 drives. The default for the tape jobs looks to be set to 2 (it’s unchecked, but the value shows 2). I could set it to 3.

Even so, not being able to skip sending a specific full to tape complicates things. I may open a ticket and ask support if there’s a way. Otherwise, if I miss one weekend, it will NEVER have enough time to catch up. And that can’t be right, not for an enterprise level program …

Thanks

Tick off the box and set the job to use all 3 drives.  Hopefully that helps get things over faster to complete for you.

Chris Childerhose - VMCA2022 | VMCE2023 | Veeam Vanguard 7* | Veeam Legend 3* | vExpert 5* | VCAP-DCV/VCP-DCV | Object First Ace | Cisco Champion | Twitter: @cchilderhose | Blog Site – https://just-virtualization.tech
JMeixner
Userlevel 7
Badge +17

When it’s unchecked, the setting is not active...

Jochen (Joe) Meixner | Veeam Vanguard 2024 | Veeam Legend 2021 - 2023 | Veeam Certified Architect (VMCA) 2022 | Twitter: @JoMeix | BlueSky: @jmeixner.bsky.social
M
Userlevel 4

Veeam has to make sure that the complete backup chain is written to tape. You cannot restore from the incrementals when then full from last Friday is missing.

So, you have to let the job finish. And yes, this can cause a missed incremental or an incremental at another tome of the day.

Do you have one tape drive only? If you have more, you can try to configure the tape job to use multiple drive in parallel to get the data faster to that tapes.

I have 3 tape drives. And one media pool, which is set for parallel processing to use up to 3 drives. The default for the tape jobs looks to be set to 2 (it’s unchecked, but the value shows 2). I could set it to 3.

Even so, not being able to skip sending a specific full to tape complicates things. I may open a ticket and ask support if there’s a way. Otherwise, if I miss one weekend, it will NEVER have enough time to catch up. And that can’t be right, not for an enterprise level program …

Thanks

Comment


Terms & Conditions