File Copy is very slow

Are you using a physical proxy or a VM?

What are your network speeds?

Can we have a bit of info about the source and destination?

180GB isn’t that much, even on a 100Mbit/s connection you are looking at 4.5-5 hours.

Hmm...agree with Scott here. Shoudn’t take that long. I’m wondering @GHuber ..do you have a Backup Proxy configured, as well as a Cache Repository? See the diagram of how File Backup works here.

The File-Copy runs within an VMWare ESXi-Host (Free Edition)

• source files are located on a virtual machine named “FILE-SERVER”
• destination is a physical disk in the same ESXi-Host.
• let’s say ‘the files are copied from one raid-system to another one, both are located in the same ESXi-Host”

There is a backup-proxy named “VMware Backup Proxy” - i’m not sure how to assign this proxy to the mentioned copy job.

my network speed is ‘gigabit speed’ - but it is not relevant here - because the process runs ‘within’ the ESXi host

Well that is relevant, as if you choose the wrong transport mode it can use the VMware Management network which is often slower on most hosts.   

It looks like this is your homelab, but the community will try and help you, these issues get resolved very fast when you have a support contract.

Unfortunately, there are many things that could be the issue. You said same physical disk. Is this an SSD, NVMe, or slow disks? how many VM’s are running on this disk, or raid group? if you are running VMware, File servers, Veeam, and doing heavy copies, this could actually be a disk issue and not even your network. Have you looked at the resources from VMware, from within the VM? disk latency using performance monitor could indicate many things as well. 

• let’s say ‘the files are copied from one raid-system to another one, both are located in the same ESXi-Host”

 Are they copied from 1 raid group to another? When you stated “Lets say” I just want to confirm this is actually happening.  Even so, I’d still recommend checking all the resources and disk latency / I/O, and making sure that isn’t the issue.

99% of the time the issue is config. Veeam will saturate my network / fiber usually as long as you keep throwing tasks at it. Concurrency is key.

if that is 160k files, that’s a lot of small files. the average file size is approximately 1.125 MB. You are going to be doing lots of reads and writes on those disks adding to the time. Try zipping a bunch of them and seeing if the performance increases. 

The Default Proxy is the VBR server itself. I recommend, if you can, to create another VM then add to your VBR > Managed Servers > add VMware Proxy. Then, configure your File Backup job to use this off-VBR server as your Proxy.

To add a Cache Repo...since you’re just testing...if you have 50GB or so available on the Proxy you create, add this space as a Repository in Veeam. Then, in the File Backup job settings, select this new Repo as your Cache Repo. For how to, see here.

Have to agree with Shane here as best practice is to add another server to be the Proxy and not the VBR server itself.  Allow it to do its job of just controlling everything.  Also when adding a proxy for file shares select this type -

Hi,

Can I clarify some points here please:

You say you’re copying around 168k files, are you doing this via File Copy - User Guide for VMware vSphere (veeam.com) or via NAS Backup - User Guide for VMware vSphere (veeam.com)?

My understanding is that File Copy wasn’t created for the purpose of copying huge volumes of files, as enumerating all of that data would take a while, and I don’t recall seeing any performance tuning being available to scale up/out the job easily. Are you trying to create a backup of these files or just have a copy in another location?

Good point & good catch Michael. In re-reading his original post, I think he does make it clear it is indeed a File Copy, not a File Backup.

I didn’t even notice he was using the Veeam server as the proxy. 

Create another server as a proxy for step 1.

Out of curiosity, what happens when you try and robocopy those files to the new location. I am just wondering how the performance is.  If it works good,  the next thing I’d do is try and zip them and use the file copy job. If it’s slow now it could be a config issue still.

Yeah, after Michael’s post I read it again as you did.  Not a great way to move files for sure.

I have to agree with the above, Veeam’s "File Copy” feature is an overly complicated way transfer some files. It may be simpler from a user perspective to do that if it’s convenient, but as far as the software that's doing it is concerned, it's far more complicated than just a basic Windows Explorer file copy, which is probably why it’s taking so much longer. If you must use Veeam, use something else (PowerShell, Windows Explorer, 7-Zip) to ZIP the files first, so Veeam only has 1 actual file to transfer, then you won’t be slowed down by Veeam’s overcomplication of the file copy process so much and it’ll come down more just to data transfer speeds on the hardware.

Thanks all together - it seems to be as mentioned in some posts indeed. The ammount of 160k files - some of them are small - seems to be the issue. 

Hi @GHuber -

I’m just following back on your file copy post. I just want to verify one of the above comments helped you answer the slowness of your file copy process? If so, we ask you select which comment best helped as ‘Best Answer’ so others with a similar question who come across your post may benefit.

If you have further questions, don’t hesitate to ask.

Thank you!

I gave a like to scotts answer - cant find a way to mark “as best answer”

You did it correctly. 😊 Thank you for following up. 

Funny this came up today. I have a folder I am dealing with that isn’t huge, but a few million files.  Even RoboCopy struggles with things like this.  A 25Gb network doesn’t matter when your I/O and CPU bound. lucky for me RoboCopy with the /MT (multithread) switch helps a ton.

Veeam NASBackup and File to Tape both also benefit from zipping things up. if you want to get those tapes streaming full speed.

Another option you could do is run many smaller jobs on folders than one large job. It’s all about concurrent tasks if you aren’t maxing things out.