LAN connection speed

[Bill Hassell]

> Keith:
> 1) Try ftp with a small block size ftp -B 1
> 2) Lower the block size on your ftp daemon in inetd.conf .

This doesn't match my experience and is counter-intuitive to the concept that small block sizes will require a much greater percentage of system overhead to handle the extra I/O requests. Using ftp -B 10000, a 500 Mbyte file transfer was significantly faster. Conversely, ftp -B 1 was not significantly different than using the default. This is probably due to the network driver coalescing small packets to reduce I/O counts.

> 2) If any network component in the data patch has "Jumbo Packets" enabled this can be a performace disaster for ftp. It is a particular problem in mixed manufacturer networks (e.g. HP and Cisco).

Again, larger packets means lower overhead, but this assumes that all the components along the network path understand jumbo frames.

[Keith Jahn]

@Bill Hassell
These tips are based on real world problems where FTP runs slow when it should run fast.

Lowering the ftp block size and buffer size can cure this issue where the target is a Microsoft platform or even a DEC unix server.

There are real problems with FTP negotiation on modern network hardware and a connection which might work perfectly on a 100 Mbit/s hub can run like a dog on cascaded 1000 Mbit/s switches.

[Kevin (Gonzo) Bushman]

All - I appreciate your replies and I've continued to research my issue taking some of your replies into account. Here's what I've found.

- Patrick is right - you have to set it to auto to get GB speeds. As shown, mine are set correctly. As I also stated, once I found to look at the (ahem) RIGHT NIC, it was connected at 1GB.

- Block size tests. I've run some sample FTP's between the various systems (all HP's, some at 11.11 - the source for example, some at 11.31 - the destinations mostly). The FASTEST I've been able to transfer is 26.5MB at the default 32768b block size and most transfers were only at 10-15MB/sec. I then ran additional transfers at both 10,000b and 50,000b block sized to run at both small and large block sizes (sftp -b 10000 and sftp -b 50000). These times were slower which resulted in slower transfer speeds. They also tended to stall more often.

- Load on the systems. The source system is fairly heavily loaded. That is, the CPU's are generally pretty busy (50-60% of capacity on average), so that MAY be a factor. I don't think memory is an issue as this system generally runs at about 60-65% of physical in use, and with "only" 16GB total (sorry, it's an old machine), that still leaves 6GB or so free. The destination systems on the other hand are not even in use yet. And they are 11.31 systems with much more memory. Also, I ran a speed test early Sunday morning when the source system wasn't busy (10% CPU utilization generally) and got the same speeds.

So, what else are we missing? Why can I not get anything better than 10-15% of the card's capacity (10-15MB/sec = 100-150Mb/sec, roughly or 10-15% of 1Gb)? What is preventing this from working at the rated speed? Could it possibly be as simple as the fact that I'm running sftp, not ftp?

Any ideas guys/gals?

-Gonzo

[Patrick Wallek]

SFTP could definitely have an impact due to the overhead required for encryption.

Would it be possible for you to try a test with straight FTP (not SFTP) and see what kind of speeds you get?

[Bill Hassell]

sftp is not ftp -- it is a completely different product and is actually slower than scp. ftp actually has a number of design features such as sending multiple blocks and not waiting on each block for an acknowledgment.

If CPU were a problem, there would be 100% usage by the sftp process. And memory would not be an issue as sftp (and ftp) don't grab big chunks of RAM for execution.

I would try ftp -B 1000 host_name just to see what the results are. I have also seen that multiple ftp sessions (3-5 total) going in both directions can exceed a single session throughput. You might use lanadmin to clear the stats and then count the results (lanadamin -g 1) when running several copies at the same time.

[rick jones]

What is netstat -s -p tcp telling you when bracketing an FTP (or better still netperf TCP_STREAM) test?

netstat -s -p tcp > before

netstat -s -p tcp > after
beforeafter before after > delta

where beforeafter can be had (for the time being, the server will be going away eventually) from ftp://ftp.cup.hp.com/dist/networking/tools/

[rick jones]

Keith - what exactly do you mean by "ftp negoitiation?"

[Kevin (Gonzo) Bushman]

So I did as suggested and tried some plain ftp's. Here's the results:

56 buffers (default) -> 15.65MB/sec
10 buffers -> 15.05MB/sec
64 buffers -> 14.94MB/sec

Even considering overhead, this is still only in the neighborhood of 150Mb/sec, far short of the 1Gb card's capacity. The destination system was all but idle. The source system was at about 50% CPU (on 8 proc's) and memory at about 63%.

-G

[Shibin_2]

I guess, enough testing at system side. How many devices connected to the switch, where this server connected? Have you tried to replace the cable with new one or with anyone that delivering better speed to others?

[rick jones]

If his cable was bad that should have shown-up as errors of some form or another in the lanadmin statistics.