Question

Nimble HF40 as backup target


Userlevel 1
Badge

Hi floks,

does anyone use a Nimble HF40 as backup target for Veeam. So backup repository is configured als LUN formated with ReFS. We face really bad performance during tape jobs. After all it seems, Nimble Storage is not able to deliver more than 380 MB/s. With one tape drive configured to a job, the calculated processing rate within this job is 152 MB/s. Really bad performance...


11 comments

Userlevel 5
Badge +1

Hello @stadi13 ,

which kind of Tapedrives do you have?

We have some LTO-6 Drives and the datarate is around 150 MB/sec per drive, too. But the specification for LTO-6 says 160 MB/sec datarate max (uncompressed).

Userlevel 1
Badge

It’s LTO-8 so up to 300MB/s per drive should be possible whereas 280MB/s is mostly seen.

Userlevel 7
Badge +2

Just to try and understand better, you are saying the performance issue is specific to the backup to tape jobs? Would it be possible to elaborate a bit more on the setup?

  • Which server is the backup repository?
  • How is the repository connected to the Nimble HF40? FC? iSCSI?
  • What is the tape library connected to? Same server as the backup repository or a different server?
  • If tape server is a different server than the backup repository, how much bandwidth do you have between them? 10 GbE?
Userlevel 6
Badge +2

I have seen tape-backup performance far away from what LTO-generation should be deliver. And most often, source storage was not the problem. 

Have you tested the performance of your Nimble array? You can install a tool like IOmeter to perform basic performance tests on the array. So you can reduce the sources of your performance issue.

Userlevel 7
Badge +2

@vNote42 Which IOmeter settings would you recommend for testing this scenario?

Userlevel 6
Badge +2

@vNote42 Which IOmeter settings would you recommend for testing this scenario?

 

Honestly, hybrid arrays like Nimble are difficult to performance-troubleshoot. In case of backup-repository, you would like to test spindle speed. 

Usefull would be:

  • IOmeter with: larger blocksize, sequential, read. This will primary delivered from SSD, but you can see if there is a basic-issue with the communication between array and Windows respectively with the array itself.
  • Robocopy (or similar) to copy backup-files from Nimble to local disks. This should be delivered from spindles.

These steps could be done easily without changing Nimble configuration.

Userlevel 5
Badge

@vNote42 Which IOmeter settings would you recommend for testing this scenario?

 

Honestly, hybrid arrays like Nimble are difficult to performance-troubleshoot. In case of backup-repository, you would like to test spindle speed. 

Usefull would be:

  • IOmeter with: larger blocksize, sequential, read. This will primary delivered from SSD, but you can see if there is a basic-issue with the communication between array and Windows respectively with the array itself.
  • Robocopy (or similar) to copy backup-files from Nimble to local disks. This should be delivered from spindles.

These steps could be done easily without changing Nimble configuration.

I’d also just add that as Infosight monitors all of the Nimble metrics continuously, too much testing will then also cause HPE to claim you’re low on cache etc and start trying to push an upgrade on you.

I deployed a new backup repository to a customer and because they benefited greatly from ReFS I didn’t want to migrate the data as it would’ve rehydrated the blocks, the customer agreed we’d just cut over to new chains and age out their old ones. I performed active fulls of all VMs as I felt it was a good opportunity to do so, the customer had an unrelated issue with their Nimble nearly a week later where IOPS fell to about 10IOPS, randomly bursting up to normal traffic levels then dropping again. Nimble support kept repeating that based on the poor amount of cache hits that cache needed nearly doubling and that the customer needed to step up two tiers of hybrid Nimble, purely because of all those rarely used blocks I read for the active backup.

For reference, the customer’s issue was that the Nimble had stopped seeing all of its NICs properly and was intermittently showing me the interfaces for one controller. A manual controller failover resolved it as it rebooted the controller that was freaking out, but Nimble refused to do anything else, disappointing as I’ve always been a huge advocate of their Infosight platform and support.

the LTO Speed is max. 300MB/s uncompressed so 280MB/s is quite good , i think youre main problem is the Nimble, this system is not suitable for use as backup target especially you want to create regularTapeouts. The hf40 is a quite good storage, but if you create a backup to disk and a tapeout is created at the same time. The speed will not be very fast. Check the infosight portal.

Badge

the LTO Speed is max. 300MB/s uncompressed so 280MB/s is quite good , i think youre main problem is the Nimble, this system is not suitable for use as backup target especially you want to create regularTapeouts. The hf40 is a quite good storage, but if you create a backup to disk and a tapeout is created at the same time. The speed will not be very fast. Check the infosight portal.

This behavior is normal, since you do sequential write and seqential reads at the same time.
Nimble is always good at random workloads. But concurrent sequential workloads can bring some trouble. While other arrays scale with disk count, Nimble doesn´t. It is CPU centric. Try to avoid concurrent sequential read/write workloads as there is no caching for reads. (See CASL whitepaper)
Maybe backups at night and tapeout over the day? Does this fit in your backup-windows?

Userlevel 6
Badge +2

You probably checked it, but there is a special performance policy for Veeam Backup in the Nimble configuration. Did you select this for your repository volumes?

Userlevel 6
Badge +2

You probably checked it, but there is a special performance policy for Veeam Backup in the Nimble configuration. Did you select this for your repository volumes?

PS: 

 

Comment