How to increase one-to-many simultaneous LAN file transfer speeds


#1

Hello -

This is probably a pretty basic question, but here goes…

In a small office domain network, we typically don’t need to transfer a lot of large files. But, now there is a need to allow around 20+ clients to all copy a large file (around 20+ GB) at the same time and minimize how long each download takes to complete.

All PCs and LAN are currently equipped for 1 Gbps.

Right now, if one client downloads the file it only takes about 3 minutes to complete. However, (not surprisingly) when multiple clients try to download at the same time each additional download takes significantly longer.

I’m wondering what the shortest path (and ideally cheapest) would be to improve this. Are hardware upgrades required? Some sort of specific/dedicated file server? Any software that can achieve this with the existing hardware? Etc…

Any suggestions or links to other good existing references for this sort of thing would be much appreciated.

Thanks!


#2

Have you looked into multicasting? Something I learned about back in networking school but have never had to apply:

http://www.enterprisenetworkingplanet.com/netsp/article.php/3623181/Networking-101--Understanding-Multicast-Routing.htm


#3

There are different ways to approach this:

First, you can try a multicast file transfer program. This would require each of the clients receiving the file to set up some sort of listener program and wait for the server to send the file. There is a file transfer protocol known as UFTP that does this:


also, look at this program:
http://uftp-multicast.sourceforge.net/

Another way is to increase the bandwidth and efficiency of the server.

  • Install a 10Gbps network card and upgrade your switch with at least one 10Gbps port. This will allow 10 clients to receive the file at maximum speed, or 20 clients to receive the file at 500Mbps.
  • Add enough memory that the entire file can be cached in RAM
  • Install an SSD for this file transfer. This reduces seek time and will allow the system to spend more time transferring the file, rather than seeking 20 different sectors at once.

#4

Thanks for your responses. Seems like exactly what I was looking for.

I will check out UFTP and also consider those limited hardware upgrades.


#5

UFTP is definitely the type of solution I was looking for. I think this will work.

I am still working through two issues:

  1. When I push files out over UFTP I notice what seems like other bandwidth-related issues (e.g., VoIP phones reboot, wi-fi drops, server connections drop,etc.). I already have some bandwidth management in place (at least for VoIP, which should always have enough reserved), so I’m wondering if UFTP is somehow working outside of those rules… not sure.

  2. I can’t get UFTPD to run as a service on Windows 10 Pro. I get “Error 2: The system cannot find the file specified.” (see attached screenshot). This seems like it would be a pretty common issue to identify but so far my searches have come up empty for a fix.

Any additional advise on the above would be appreciated.

I always want to contribute properly, so if these questions should be started as a new thead(s), please let me know. I read the FAQ but wasn’t entirely sure if I needed to start new or not.

Thanks.


#6

I don’t have experience using multicast, but it is a type of broadcast. So even though you have QoS in place, it might still be broadcasting where it shouldn’t be. I wonder if you could use vlan’s to try and isolate it.

As for the Windows 10 error, its possible you need to make sure a certain port is open in the firewall on the system.

https://support.microsoft.com/en-us/help/883831/a-client-may-not-be-able-to-connect-to-multicast-streams-or-to-other-u


#7

Yeah, the thing to do here is slow the broadcast rate. Hopefully the UFTP server has some sort of rate throttling to slow down the transmission speed.


#8

I tried turning Windows Firewall off completely but the service still doesn’t start. I also notice I was missing a “” in the path in the registry setting, but still having the issue after correcting that typo.

Yes, UFTP does allow you to set the transmission speed (using -R). I am setting it to 50% of my bandwidth but still seeing those behaviors. Timing of when I’m testing UFTP transfers seems to suggest UFTP is the cause but I can’t pinpoint or confirm that for sure so far.


#9

You might check the firewall on the router, but I’m fairly certain that would only apply to traffic entering and leaving the local network, and I’m assuming this is a multicast over the local network.

Did you see this?

https://social.msdn.microsoft.com/Forums/sqlserver/en-US/b803a3b0-559f-4a1b-ab44-4bb64e0c746d/error-2-the-system-cannot-find-the-file-specified-windows-event-log-service-could-not-start?forum=winserver2008appcompatabilityandcertification

When you tested in on Windows 10 pro were you logged in as admin, or as a user with limited access? Are the computers part of a domain?

Since you are using VoIP, I’m assuming you have a typical setup where the ethernet cable runs from the wall to the phone, and then another ethernet cable goes from the phone to the computer. Did you try removing the phone from the equation to see if that would make a difference?


#10

Yes, I’m doing this only over LAN.

Yes, I am logged in as local admin and computers are joined to a domain.

No, computers network connections are not being passed through the VoIP phones.

Thanks for the link. I hadn’t found that one and it’s very similar to my issue but the solution doesn’t seem to help me b/c I don’t have the extra registry entry that they describe. But I did download procmon and procexp and will try to troubleshoot further with those tools.


#11

Since the computers are on a domain have you looked at group policy in reference to multicasting?

https://computerstepbystep.com/turn-off-multicast-name-resolution.html

Then again it may not be a thing:


#12

Fixed. UFTP readme file says to copy instsrv and srvany into system32 folder. That wasn’t working. Move them to some other folder (C:\whatever), and it works. No more Error 2. Thanks all for the help!