topic Re: Slow FTP performance in one direction in Operating System - OpenVMS

Slow FTP performance in one direction

Stephen Eickhoff_1 — Mon, 06 Oct 2008 19:00:16 GMT

I have a DS10L using the stock DE504-BA 100Mbit controllers. It runs VMS 8.2 (Update 11) with TCPIP 5.6 E2. The controllers are configured with Failsafe IP. WE0 is the home interface and is currently active.

If I FTP a file of about 8 MB from this server to a known-good Windows server (known good in that it transfers at over 9 MB/s to other hosts), it transfers at only 1.12 MB/s. Downloads are another matter; they go at an acceptable 9.52 MB/s. The card is currently running at 100/full using autonegotiate mode on an unmanaged Netgear switch. The statistics show no collisions or errors.

Is this expected behavior with the DE504, or should I be looking for other causes?
$ mcr lancp show dev /count

Device Counters EWA0:
Value Counter
----- -------
6066138 Seconds since last zeroed
115712493341 Bytes received
95062349392 Bytes sent
111344378 Packets received
95229158 Packets sent
416811081 Multicast bytes received
408785380 Multicast bytes sent
4735536 Multicast packets received
4797404 Multicast packets sent
4 Unrecognized unicast destination packets
2024034 Unrecognized multicast destination packets
0 Unavailable station buffers
0 Unavailable user buffers
0 Alignment errors
0 Frame check errors
0 Frame size errors
0 Frame status errors
0 Frame length errors
0 Frame too long errors
0 Data overruns
0 Send data length errors
0 Receive data length errors
0 Transmit underrun errors
0 Transmit failures
8 Carrier check failures (28-JUL-2008 10:18:53.62)
0 Station failures
0 Initially deferred packets sent
0 Single collision packets sent
0 Multiple collision packets sent
0 Excessive collisions
0 Late collisions
0 Collision detect check failures
3 Link up transitions (28-JUL-2008 10:18:53.64)
1 Link down transitions (28-JUL-2008 10:18:52.62)
28-JUL-2008 09:29:19 Time of last generic transmit error
None Time of last generic receive error

Device Counters EWB0:
Value Counter
----- -------
6066138 Seconds since last zeroed
922962921 Bytes received
851293627 Bytes sent
8860963 Packets received
8572720 Packets sent
448612703 Multicast bytes received
376986720 Multicast bytes sent
4910485 Multicast packets received
4622379 Multicast packets sent
0 Unrecognized unicast destination packets
2024036 Unrecognized multicast destination packets
0 Unavailable station buffers
0 Unavailable user buffers
0 Alignment errors
0 Frame check errors
0 Frame size errors
0 Frame status errors
0 Frame length errors
0 Frame too long errors
0 Data overruns
0 Send data length errors
0 Receive data length errors
0 Transmit underrun errors
0 Transmit failures
6 Carrier check failures (28-JUL-2008 10:20:13.82)
0 Station failures
0 Initially deferred packets sent
0 Single collision packets sent
0 Multiple collision packets sent
0 Excessive collisions
0 Late collisions
0 Collision detect check failures
3 Link up transitions (28-JUL-2008 10:20:14.32)
1 Link down transitions (28-JUL-2008 10:20:12.82)
28-JUL-2008 10:20:13 Time of last generic transmit error
None Time of last generic receive error
$ ucx show int /fu
Interface: LO0
IP_Addr: 127.0.0.1 NETWRK: 255.0.0.0 BRDCST:
MTU: 4096
Flags: UP LOOP NOARP MCAST SMPX
RECEIVE SEND
Packets 13052874 13052874
Errors 0 0
Collisions: 0

Interface: WE0
IP_Addr: 192.168.0.25 NETWRK: 255.255.255.0 BRDCST: 192.168.0.255
Ethernet_Addr: 08-00-2B-87-0D-74 MTU: 1500
Flags: UP BRDCST RUN MCAST SMPX
RECEIVE SEND
Packets 102790766 86610704
Errors 0 4
Collisions: 0

Interface: WE1
IP_Addr: NETWRK: BRDCST:
Ethernet_Addr: 08-00-2B-87-0D-75 MTU: 1500
Flags: UP BRDCST MCAST SMPX
RECEIVE SEND
Packets 262044 0
Errors 0 0
Collisions: 0

$ mcr lancp show dev /char

Device Characteristics EWA0:
Value Characteristic
----- --------------
1500 Device buffer size
Normal Controller mode
External Internal loopback mode
08-00-2B-87-0D-74 Default MAC address (Hardware LAN address)
Multicast address list
Ethernet Communication medium
FF-FF-FF-FF-FF-FF MAC address (Current LAN address)
128 Minimum receive buffers
256 Maximum receive buffers
Yes Full duplex enable
Yes Full duplex operational
08-00-2B-87-0D-74 MAC address (Current LAN address)
TwistedPair Line media type
100 Line speed (mbps)
Enabled Auto-negotiation
Disabled Flow control
Disabled Jumbo frames
Disabled/No Failset Logical LAN state
0 Failover priority
Link Up Link state

Device Characteristics EWB0:
Value Characteristic
----- --------------
1500 Device buffer size
Normal Controller mode
External Internal loopback mode
08-00-2B-87-0D-75 Default MAC address (Hardware LAN address)
Multicast address list
Ethernet Communication medium
FF-FF-FF-FF-FF-FF MAC address (Current LAN address)
128 Minimum receive buffers
256 Maximum receive buffers
Yes Full duplex enable
Yes Full duplex operational
08-00-2B-87-0D-75 MAC address (Current LAN address)
TwistedPair Line media type
100 Line speed (mbps)
Enabled Auto-negotiation
Disabled Flow control
Disabled Jumbo frames
Disabled/No Failset Logical LAN state
0 Failover priority
Link Up Link state

Re: Slow FTP performance in one direction

Richard Whalen — Mon, 06 Oct 2008 19:47:25 GMT

I would suggest getting a TCPDUMP and looking a the TCP window size in each of the transfer directions.

Re: Slow FTP performance in one direction

Hein van den Heuvel — Mon, 06 Oct 2008 19:57:30 GMT

>> If I FTP a file of about 8 MB from this
server to a known-good Windows server
(known good in that it transfers at over 9 MB/s to other hosts), it transfers at only 1.12 MB/s.

Please confirm that the performance problem is copying from VMS to Windows and not the other way arouns. Typically copying to OpenVMS "out of the box" is slow due to small RMS defaults such as small file extends, too few and too small buffers.

Please confirm that this 'known to be good' host can receive FROM other hosts at 9mB/sec.

>> The statistics show no collisions or errors.

Unless this transferring is the only thing those servers ever do (I doubt it) Please consider zero-ing the counts and then try.

The counters have been going got a good while (100 days?) and surely have seen all sorts of activity happen.

> Is this expected behavior with the DE504, or should I be looking for other causes?

Going un-tuned towards OpenVMS yes. From no.

fwiw,
Hein.

Re: Slow FTP performance in one direction

Hoff — Mon, 06 Oct 2008 20:01:24 GMT

Start toggling some stuff. Duplex or not. Active controller or not. Connected or not.

The IP setting on WE1 looks odd.

And if you want speed, jumbo frames (if the network can deal with it) helps. (Do investigate the "2024034 Unrecognized multicast destination packets" here, too.)

Post up:
LANCP> show device /internal_counters
and
LANCP> show configuration /users, too.

Weird stuff can stomp on this; bad or slow DNS, duplex mismatch, software bugs, switch errors, etc.

Might be worth zeroing the counters before running another test; this so we're not chasing old data.

Re: Slow FTP performance in one direction

Jur van der Burg — Tue, 07 Oct 2008 05:44:40 GMT

As Hein says, confirm that the problem is copying TO VMS. That is the most common thing seen, and caused by the rediciously low default rms extend size. Try a SET RMS/EXTEND=xxx or SET VOL/EXTENSION=DKxx.

Jur.

Re: Slow FTP performance in one direction

Robert Gezelter — Tue, 07 Oct 2008 07:22:50 GMT

Stephen,

I concur with Hein, Hoff, and Jur. First, check that the RMS settings for extend are fairly large (enough to accommodate the entire file). These settings can be set in the user's LOGIN.COM or on a global basis.

If that does not clear the problem, my suspicion would also be on how things are moving on the network. In particular with mismatched duplex settings, the precise timing behavior can result in strange timing problems and odd performance. Note that a LAN sniffer (e.g. WireShark) can be invaluable in showing what IS ACTUALLY happening on the wire, as opposed to what one THINKS MAY be happening on the wire.

- Bob Gezelter, http://www.rlgsc.com

Re: Slow FTP performance in one direction

Wim Van den Wyngaert — Tue, 07 Oct 2008 07:43:09 GMT

http://h71000.www7.hp.com/doc/83final/6526/6526pro_041.html#trouble_ftp_perform_sec

Try setting these logicals.

Wim

Re: Slow FTP performance in one direction

Jim_McKinney — Tue, 07 Oct 2008 10:19:42 GMT

Each of the items noted by other posters influence transfer speed - however, if the the slowness is only outbound and only to this particular host I'd take Richard Whalen's initial advice and look at the negotiated TCP window sizes. Mismatched window sizes will result in abismal performance. You'll want the send window size on the VMS system to match the receive window size on the Windows system. The tail end of the following thread discusses this phenomenon - http://groups.google.com/group/vmsnet.networks.tcp-ip.multinet/browse_thread/thread/d80a8fe487016de0/ .

Re: Slow FTP performance in one direction

Wim Van den Wyngaert — Tue, 07 Oct 2008 11:20:47 GMT

The window size should be very low to explain those speeds. U found these (old) test results.
***
Default is 64K windows and thus the sender will transmit at high rate, the receiver will ack e.g. every 100 packets.

With 1.5K windows, speed is reduced to 50% but overhead is higher because every packet is acked.

With 0.5 windows, speed is reduced to 30% but still more overhead.
***
30% still doesn't explain your results.
May be something is pumping at the same time as your ftp (in any protocol, also check decnet etc).

I would test with a bigger file too.

Wim

Re: Slow FTP performance in one direction

Stephen Eickhoff_1 — Tue, 07 Oct 2008 12:57:20 GMT

Status update:
Hoff, this is a 100Mbit adapter, so jumbo frames don't apply.

The slowness is from VMS to the Windows machine, no the other way around. Regardless, I found that the Windows server is at only 128K while VMS is 512K, so I will boost the Windows server.

I ran Wireshark to look at my traffic. It seems that the high multicast counts may be due to Failsafe IP keep-alives. I may try disabling Failsafe to see if anything changes.

I reset the counters, and that didn't tell me anything new. There were no errors before, and none after.

DECNET is not even installed.

The transfers are not so slow as to suggest a duplex problem. I have seen those many times before, and the result is more like 5 MB/s in one direction and 10KB/s in the other.

I have to reboot the Windows server to get its window size to change, so I tried changing VMS to 128K instead. Strangely enough, the speed to Windows now jumped to over 9 KB/s, but VMS dropped to 1.1KB/s! So I checked my RMS settings. I found that I had this in my login.com, but I had commented it out for some reason:

set rms/seq/block=127/buf=8
set rms/ind/buf=20
set rms/extend=4096

I applied those and ran the test again. I got 2.33MB/s. I can't see why my RMS settings were okay before but not now. I'm starting to think I ran my earlier tests improperly. My DS10L is running an IDE disk, so I don't expect to get much more than this out of it.

Re: Slow FTP performance in one direction

Stephen Eickhoff_1 — Tue, 07 Oct 2008 14:36:20 GMT

TESTDEV confirms that the max sequential write speed of this disk is 2.4MB/s. Yuck. Maybe I should see if I have a non-whiny 1/3 height SCSI disk to replace it.

Re: Slow FTP performance in one direction

Stephen Eickhoff_1 — Tue, 07 Oct 2008 17:25:36 GMT

Now downloads at near disk speed with RMS and FTP window size tuning.