No, we are using physical libraries only.

I don’t see much advantages in using a VTL. The data is in files in a filesystem which is not immutable normally. So, The big advantage of tape - the airgap - is gone.
So, for testing purposes… maybe

Ok, you can use some specialized machines like an IBM TS7700, but then you can use a normal library, too….

I had exactly one customer using an AWS VTL, and that was because archive tier object storage integration didn’t exist yet, I then migrated them to object storage and all was good! This was a leap from v9 to v11, so again, very specific scenario.

I had a VTL gateway setup to AWS back in the day. I also played with Quadstor https://quadstor.com/virtual-tape-library.html 

I believe you could actually export the VTL tape to real tapes as well. 

Tape is Airgap only in the physical understanding, however if going by the strict definition of  Airgap as per the official NIST definition you cannot have automated transfer of data, it must be manually performed by humans, so scheduled Veeam jobs don’t fit the definition.

https://csrc.nist.gov/glossary/term/air_gap#:~:text=Definition(s)%3A,manually%2C%20under%20human%20control).

rfc 4949

I have seen some marketing departments using the term air gap a lot even for Veeam’s insider protection but it is not correct.

AWS VTL is something different but this was not the question if I understand it correct.

We currently do not use VTL with Veeam but rather physical tape libraries at the moment.  I know we are investigating VTL but not sure if that will be for Veeam specifically or other services.

No, same thought as @JMeixner @Chris.Childerhose @Geoff Burke 

i don’t understand the need for Veeam, maybe a protocol break from tcp/ip to FC but it will make more sense on physical tape library.

There are a number of good posts on using Quadstor with Veeam if you want to test. Very quick and easy to setup and it can emulate real drives:

https://www.veeam.com/blog/how-to-create-vtl-and-why.html

https://jorgedelacruz.uk/2018/10/16/veeam-quadstor-a-perfect-option-for-linux-based-virtual-tape-library-vtl-free-and-very-easy-to-deploy/

https://domalab.com/tape-backup-veeam-quadstor/

Yes, all of these are really great resources.  I actually started looking at them this week. 

We do use VTL which does work quite well for our use case. It is quite robust and there is the option to have WORM only media so it can not be overwritten, together with expiry options when using GFS. 

The tapes can be taken ‘offline’ and stored into another archive tier. However, recovery times can be quite large when a tape is recalled. 

No VTL here.  Honestly, the only time I’ve seen VTL in use was in my old environment for our IBM I Series (AS/400) and Z-Series (Mainframe) folks so that they could backup to our Avamar/DataDomain environment as if they were tapes.  Otherwise, I’ve never seen a need for it.

Thank you all for all this answers !

I share the same point of view as some of you. I only implement physical tapes libraries cause there is no real Air-Gap with a VTL. A reason for a VTL implementation could be the one mentionned in this blogpost https://www.veeam.com/blog/how-to-create-vtl-and-why.html, where a tape job doesn’t work like a backup copy job.

I don’t know AWS VTL but what is the difference with a VTL created on a Datadomain for example ?

@dips  what is your use case if it’s not confidential ? 

I just set up CentOS with Quadstor VTL so I have tape server on Linux for v12 and Chapter 4 of my book.

I have tested a VTL some years ago (not with Veeam, but with another backup product).

The functionality is very similar, tthis s no problem.

Our painpoints were:

• there is no physical separation of the media like with physical tape libs.
• the throughput was not the same. With a physical library each tape drive has a own fibre channel connection. With a VTL all drives share the network connection(s) - this is much less in most cases, at least when you have a library with several drives.

Nice One ! For test environnement it’s a good idea 

Last time I touched a VTL was doing hardware support on an IBM Mainframe haha

I would like to see more specifics on why someone chose not just physical tape over a VTL for security reasons, which can more obvious, but instances of choosing a Physical Tape Library over a Virtual Tape Library. In the Latter the gap in security near evaporates or is eliminated. You would still need to protect a PTL in the general same ways as a VTL, they get access to it, then “tape” media becomes pointless without additional steps (WORM, immutability, encryption, etc.)

If you went PTL route, but are not physically taking out the most recently wrote tape every time, what product/setting/technology are you using that ensures someone can’t just gain access to the management easily gain access to that tape?

There are other ways of defining a VTL.

Are you referring to using disk as the physical media and a virtual tape in front of it?

For me, I have a few TS4500’s from IBM loaded full of drives with tapes that get ejected. I use the built in VTL functionality to make it appear as multiple libraries. Each with their own Tapes, drives and policies.  With this I am actually running both Veeam and TSM on the same PHYSICAL machine, but each assigned to their own VTL.   

Technically if the tapes are in the machine, security is the exact same.  It comes down to physical and logical access. You can have VTL’s on different subnets and environments which is nice, but at the end of the day ejecting the tapes is the ONLY 100% way, and the reason you want tape. 

VTL may be different in the IBM world, but that works great for me. I used to service giant VTL’s that were disk based and although faster, it’s really no different that backing up to SAN. You are just using a different protocol. Things like older IBM mainframes need tape to boot and rather than changing the front end you just slap a VTL on it and away you go. 

Thanks Scott. Seems like my thinking isn’t too far off and it seems we agree there is a difference in eject versus non-ejecting context, correct me if I am wrong. I just wanted to make sure others who might come by this didn’t not have a tldr; scenario and just thought the clear answer was physical tape.

We are in the mist of implementing something new ourselves and I was thinking of kickstarting it off for free by repurposing hardware and using QuadStor since it is supported by Veeam. This would fill a gap in our Backup Program. Then next year decide to upgrade the hardware and keep VTL or switch to something else (e.g. PTL w/or w/out ejecting) based on the previous year’s worth of analysis. As part of the first thoughts of this process, I was looking at ways to prevent manual intervention, to increase ease of adoption, willingness of staff and reduce human error (e.g. oops, forgot to put a new tape in), but yet still increase security a considerable amount more than doing nothing. It is all part of a cost analysis for us (yes I know an actual breach would cost more than any solution). 

I can be pretty hard to find information surrounding the security of PTL versus VTL. I’ve found very few resources recently that go into the details about the vulnerabilities of each in plain English with direct comparisons. I came across this post and wanted to try and help context to the dismissive outlooks on VTL in it.



Not to hi-jack the OP, but trying to ask the opposite of what they are in order to get pretty much the same answer:
For those that went the physical tape route, or even remove flash drives or hard drives route, did you implement a Library? How does that library handle the tape after written to, assuming non-WORM. If not ejecting, what technologies does it have to help prevent a malicious actor from gaining access to the management console, forcing the rotation of drives while wiping each one, or corrupting it in some way?

Anyone have a Physical Tape Library that has flexibility in when you can restock it with tapes and it can auto-eject tapes it a bucket or something daily? For example, something that can hold 14+ drives, ejects 1 per day automatically, and you have up to 7 days in between refills.
Asked another way, other than physically ejection tape to 100% ensure it is offline, the fact that it gives you the 2nd type of media in a 3-2-1, and that it is an option for storing something for 30 years, what other benefit does it provide that makes it a clear winner?

I would say this was the only reason to use, but now that we archive tier on VBR I don’t see any reason to use VTL.

I am using physical tape libraries in some customer environments. It's the cheapest storage for long time retention.

In one case the tapes are ejected and stored in a safe.

The management consoles are protected by user and passwords, dedicated VLANs and dedicated special protedted systems that can access these VLANs.

And yes, object storage - especially with immutability - gets more and more interesting. Prices are decreasing and the handling is easier and more comfortable.

@JMeixner Assuming just the context of ransomware protection and not some other DR situation (fire/floor/etc.), do you have anything fancy that ejects it into a bucket automatically or anything and someone just pops it back in at a certain cycle?

How do you handle the tape long term? Do you test restore it once every 1/3/5/10/etc years to ensure it will work, or do you do something else like just keep multiple copies and hope they don’t all fail? Not saying disk and VTL would be better in this regards, it would arguably be worse, just trying to get all data points. I only had to mess with tape for a couple years about 6-8 years ago and we had constant issues with reliability, as far as errors restoring/backing up goes.

You intend a robot to pick the out of the i/O station and to put them back in?

This would be nice, but unfortunately wi don’t have anything like this. We depend on helping hands in the DC…

The tapes with multi year retention are read at least every two years. Or at every tape drive exchange, then they are read and copied to new media.

But most tapes don’t have such a long retention, so they are reused in shorter cycles..

No I was just thinking something like it auto ejects in a tub, that way the following day’s isn’t blocked, or if the tape drive was specifically made where the mechanism was strong enough for 1 tape to push another out of the way. Then for putting it back it, someone does put it back in manually, but it would be nice if they had a grace period of a couple of days. That way on the weekend for example, no one has to come into the datacenter. I haven’t ever used a tape library myself, just 1 tape drive, so I don’t know if your standard tape library, like a Dell ml3 would be able to give you a grace period for refill. If anyone could give me a brand and example of that, then that would be nice. This is probably the main reason I am steering away from a physical tape library, it’s got to be easy to use so the effectiveness of the plan is high. 

I am still not sure if I get your problem completely.

It is the main purpose of a tape library to cange tape for one or more tape drives on demand of a backup application or something similar.

Most tape libraries have a so-called I/O station. Here are tapes deposited that are checked our from the library. From this an operator can collect the tapes and put them for example into a safe.

The checked-out tapes in the I/O station cannot checked-in again without manual intervention of an operator.

He would put tapes that should be checked-in into the library into the I/O station.

There are some vendors who have some extra secured mechanisms for checked-out tapes - e.g. Active Vault from Quantum. https://www.quantum.com/en/products/tape-storage/security-framework/

Like Joe suggested some vendors have some extra features. ActiveVault is one of them, from my pov is the first step.

Here is the clear documentation: TECH BRIEF: SCALAR RANSOM BLOCK AND ASSOCIATED FEATURES FOR CYBER PROTECTION (allbound.com)

To check data integrity on tape, you can use an edlm drive. You must define a policy check one time per month/year...

Scalar-EDLM-Datasheet-DS00513.pdf (allbound.com)