I’m planning on making a better writeup with my settings and some photos, but for now this is on my mind after helping someone today.
I often see Veeam setups with 1 large datastore / volume in Windows for their repository. Other times quite often it will be multiple disks spanned into a single logical Volume, lets call it D:\.
You can still get good performance like this, and the simplicity is quite nice, but can it be better? If you have some spare storage kicking around I encourage you to try this in your lab.
Run your benchmarks on this single volume (D:\) record your results and then then delete the logical and physical volume. Create 3 new equal sized volumes with matching logical volumes. Lets call them E:\ F:\ and G:\. Next run IOMeter or your favorite benchmarking tool. Make sure you are pushing the queues to see the benefits of multiple physical/logical volumes.
The next thing you can do is check and modify queue depth on the adapters and Windows server. With a large backup job, there is often many parallel tasks completing at once. Because you have fresh benchmarks, run them again and see the difference. Quite often SANs and Windows are designed for fairness, VMware is like this too. This is because there is usually many servers connecting at once where as a Veeam Repository is often the only server connected. Increasing the queue depth, and using multiple disks allows you to actually push the IOPS and throughput much harder.
In 15 minutes on a call with someone today, their benchmarks went from about 130k IOPS to 600k. I didn’t even touch the queue depth and only created a few volumes to show them for an example. The Latency was still at 1ms as well. I also had their proxy transferring at 40Gbps data as well which was much higher than they have ever seen. They were shocked when their next Veeam backup job showed them 5GB/s in the console for the first time ever.
It’s always good to monitor things like latency from within the server and on the SAN as well. Quite often the SAN is sitting at or below 1ms and Windows is swamped begging to chew up that disk. If you have a VMware environment, there is another level to the equation. I will state that if you don’t know what you are doing or the workload, sometimes it’s best to avoid messing with queue depth in VMware because things are created for fairness. In simple terms, it stops one VM from overloading the SAN, LUN, Adapter, etc. That being said, if you deal with SQL servers, more often than not you do want to prioritize them over other VM’s. This is where setting things per host, per datastore, per adapter come in to play. There is always a bottleneck, but you can choose where and for what. It’s amazing the information you can get from Resource Monitor, ESXtop, and storage.
Even on a single SAN and repository setup, creating it as a SOBR with multiple logical/physical volumes can increase your performance significantly. Just make sure each volume has the space required for large VM’s and their datasets. Also make sure there are enough disks, or fast enough disks in the array. These days, with large SSD pools, NVMe pools or large amounts of cache, sometimes things require some extra tweaking to get the full benefit out of them.
When I get time to write some of this up, I’ll show some helpful commands for monitoring and modifying these things. I’ll also include a disclaimer 😄. No matter what you do, record your settings to know what you changed, and benchmark every step of the way. Know what’s running on your systems and what workload will be running as well. Quite often, if you are not pushing your disk, you will notice no change and can potentially make things worse off.
