RMAN Plugin - Read Bandwidth Throttling?

Hey everyone,

Has anyone ever seen Veeam throttling bandwidth with the RMAN plugin?

I’m working with a really beefy Veeam environment at the moment. Source data is all on flash, 40GbE networking available, high performance RAID controller + 4x RAID 60 repositories in a SOBR.

Backups are flying for most workloads, VMware, Agent, NAS etc, but RMAN is letting me down.

For example, a Veeam Agent using an unoptimised network path via the firewall can still do over 200MBps (Firewall is the bottleneck here), but RMAN is doing 5-7MBps and it’s on an L2 network with no firewalls involved. Testing with iperf can get 6-8Gbps easily without tuning between the hosts and the backup repositories, one test with iperf between the backup repositories was giving 26Gbps, so the network really is moving quick.

When I’m going through the Veeam logs I’m finding things like this:

plugin_co|     [8b43] >> : Invoke: Network.UpdateThrottling\n{   (EInt32) KbpsReadBandwidth = 0\n  (EInt32) KbpsWriteBandwidth = 0\n }
plugin_co|     [8b43] out: Thread finished. Role: 'traffic throttler'.

OR

plugin_co|     [b876] >> : Invoke: Network.UpdateThrottling\n{   (EInt32) KbpsReadBandwidth = 7200\n  (EInt32) KbpsWriteBandwidth = 7200\n }
plugin_co|     [b876] out: Waiting till another side opens the pipe [/tmp/pipe{2a86813e-ae0c-4a1c-8dfb-a9285eaca677}] for reading

I typically see the throttling get set to zero at the end of a transfer, but during the transfer it’s being set to values such as 7200, or 5760 for read & write.

But the only network throttling I have configured is the default rule of ANY source to Internet, but this is private networking, RFC 1918 defined private subnets. I can’t help but feel this is related to the network bandwidth rule somehow, but I’ve got full object uploads running so I need to keep the bandwidth limitation in place.

I can’t find anything in Veeam’s documentation talking about this network throttling to know if this is my problem. The values don’t match with my networking throttling directly, and it raises more questions as it says KbpsReadBandwidth, not KBpsreadbandwidth, so is it capping at 7.2Mbps or 7.2MBps? as my backup jobs are typically running at 5-7MBps this would make sense if it’s actually KBps. My network throttling is set to 100Mbps, but as there’s an upload job it’s quite possible Veeam is splitting the pipe in half and granting roughly 50% resource to RMAN, thereby giving me the 5-7MBps range. As the workloads are distributed across the SOBR I can’t get these metrics so easily to see what is being uploaded. It would align with this however:https://helpcenter.veeam.com/archive/backup/110/vsphere/setting_network_traffic_throttling.html?zoom_highlight=bandwidth

When several jobs that fall into the same network rule run simultaneously, Veeam Backup & Replication equally splits the throttled traffic between them. For example, if two jobs run at a time, each job gets half of the throttled traffic. Once one job finishes, the other gets the entire bandwidth allowed by the rule.

Unfortunately network throttling doesn’t allow you to create superceding rules, just a rule of most restriction, so again, I can’t test without breaking my ANY to Internet rule.

I’ve opened a support case but I’m wondering if anyone in the community has seen a rogue setting that could be to blame here.

Thanks!