%TCPIP-E-FTP_SEND, cannot send on socket (for large files)

[Arno van Rennes]

Hi Wim,

If I disconnect the session while putting the file, I see it on the FTp directory on the IBM system as:

Volume Unit Referred Ext Used Recfm Lrecl BlkSz Dsorg Dsname
PRO894 3390 **NONE** 1 37 VB 4100 27998 PS RI.DATA.ST25024X.FTP

After a few minutes it disappears.

Regards,
Arno

[rick jones]

The FTP idle timeout is likely unrelated to the TCP level keepalives. TCP keepalives will not progress beyond the TCP layer, so FTP above it will still think the FTP control connection is "idle."

So, if it is indeed the case that the IBM system is upset that it hasn't heard anything on the control connection for five minutes, that means one either has to figure-out why the data transfer is taking longer than five minutes, or convince the IBM system to have a longer idle control timeout.

I'm not a VMS guy (just plain networking :) but on the FTP clients I've used, one can use the "idle" command to request a different idle timeout on the control connection. The FTP server retains the last word, but it might be worth a shot.

As for the speed of transfer, might be interesting to check the TCP statistics on the sending side, and receiving side if you can. Look for things like retransmissions and retransmission timeouts. I've no idea how much work it would take to compile under VMS, but "beforeafter" from ftp.cup.hp.com dist/networking/tools/ can be used to subtract one file of network statistics from another (at least the netstat format one finds under HP-UX and linux)

If you can check link-level stats that would be good as well. While transfers to the local VMS systems suggest that the link-level is OK, this boilerplate on duplex may be of interest:

$ cat usenet_replies/duplex
How Autoneg is supposed to work:

When both sides of the link are set to autoneg, they will "negotiate"
the duplex setting and select full duplex if both sides can do
full-duplex.

If one side is hardcoded and not using autoneg, the autoneg process
will "fail" and the side trying to autoneg is required by spec to use
half-duplex mode.

If one side is using half-duplex, and the other is using full-duplex,
sorrow and woe is the usual result.

So, the following table shows what will happen given various settings
on each side:

Auto Half Full

Auto Happiness Lucky Sorrow

Half Lucky Happiness Sorrow

Full Sorrow Sorrow Happiness

Happiness means that there is a good shot of everything going well.
Lucky means that things will likely go well, but not because you did
anything correctly :) Sorrow means that there _will_ be a duplex
mis-match.

When there is a duplex mismatch, on the side running half-duplex you
will see various errors and probably a number of late collisions. On
the side running full-duplex you will see things like FCS errors.
Note that those errors are not necessarily conclusive, they are simply
indicators.


Also, if the IBM system is "far away" - in terms of latency - eg ping times if there was any change in default TCP window/socket buffer sizes with the VMS change that may have affected the transfer speed. One limit to TCP transfer speed is:

WindowSize/RoundTripTime

While the IBM system is likely advertising the same window size as before, perhaps the VMS system is no longer taking full advantage - "WindowSize" here would be the smaller of the receiver's window, the sender's equivalen to the SO_SNDBUF, and the sender's congestion window.

[Stanley F Quayle]

> I've also set the passif mode from "Passive
> is AUTO (IPv4: OFF, IPv6: ON)" to "Passive is
> OFF" but also no results.

You want to set Passive to ON. This forces the transfer to use the control channel, which makes the channel appear to be active at all times.

This assumes, of course, that the remote end supports Passive...

[rick jones]

Stanley - Unless I'm mixing passives, passive in FTP does _not_ use the control channel. It simply reverses the roles of passive acceptor and active connector on the data connection.

[Arno van Rennes]

Hi Rick,

The window size could be an issue here!
With tcpdump, I see some action, at the start of the ftp-put session:
The window size on both sides is 65535 but the IBMSYSTEM keeps changing the window size.

See:
14:08:42.633574 STVMS1.3834 > IBMSYSTEM.21: S 1365984768:1365984768(0) win 65535
14:08:47.668408 STVMS1.3834 > IBMSYSTEM.21: S 1365984768:1365984768(0) win 65535
14:08:47.712351 IBMSYSTEM.21 > STVMS1.3834: S 2390338092:2390338092(0) ack 1365984769 win 65535
14:08:47.712351 STVMS1.3834 > IBMSYSTEM.21: . ack 1 win 65535
14:08:47.786565 IBMSYSTEM.21 > STVMS1.3834: P 1:64(63) ack 1 win 65535
14:08:47.820742 STVMS1.3834 > IBMSYSTEM.21: . ack 64 win 65535
14:08:47.868591 IBMSYSTEM.21 > STVMS1.3834: P 64:124(60) ack 1 win 65535
14:08:47.881285 STVMS1.3834 > IBMSYSTEM.21: P 1:15(14) ack 124 win 65535
14:08:48.022878 IBMSYSTEM.21 > STVMS1.3834: P 124:151(27) ack 15 win 65528
14:08:48.023854 STVMS1.3834 > IBMSYSTEM.21: P 15:30(15) ack 151 win 65535
14:08:48.196695 IBMSYSTEM.21 > STVMS1.3834: P 151:212(61) ack 30 win 65527
14:08:48.196695 STVMS1.3834 > IBMSYSTEM.21: P 30:42(12) ack 212 win 65535
14:08:48.281650 IBMSYSTEM.21 > STVMS1.3834: P 212:278(66) ack 42 win 65529
14:08:48.421290 STVMS1.3834 > IBMSYSTEM.21: . ack 278 win 65535
14:08:59.582958 STVMS1.3834 > IBMSYSTEM.21: P 42:67(25) ack 278 win 65535
14:08:59.630807 IBMSYSTEM.21 > STVMS1.3834: P 278:300(22) ack 67 win 65522
14:08:59.631783 STVMS1.3834 > IBMSYSTEM.21: . ack 300 win 65535
14:08:59.631783 STVMS1.3834 > IBMSYSTEM.21: P 67:93(26) ack 300 win 65535
14:08:59.778258 IBMSYSTEM.20 > STVMS1.4678: S 2390581620:2390581620(0) win 32768 14:08:59.778258 STVMS1.4678 > IBMSYSTEM.20: S 1182583056:1182583056(0) ack 2390581621 win 65535
14:08:59.819271 IBMSYSTEM.20 > STVMS1.4678: . ack 1 win 32768
14:08:59.900321 IBMSYSTEM.21 > STVMS1.3834: P 300:350(50) ack 93 win 65522
14:08:59.909109 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:08:59.909109 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:08:59.909109 STVMS1.4678 > IBMSYSTEM.20: . 2921:4381(1460) ack 1 win 65535 (DF)
14:09:00.032148 STVMS1.3834 > IBMSYSTEM.21: . ack 350 win 65535
14:09:00.717651 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:09:00.717651 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:09:03.228233 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:09:03.228233 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:09:08.739599 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:09:08.739599 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:09:20.278196 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:09:20.278196 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:09:43.842391 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:09:43.842391 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:10:31.514965 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:10:31.515942 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:11:35.224599 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:11:35.224599 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:12:38.938138 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:12:38.938138 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:13:42.651678 STVMS1.4678 > IBMSYSTEM.20: . 1:1461(1460) ack 1 win 65535 (DF)
14:13:42.651678 STVMS1.4678 > IBMSYSTEM.20: . 1461:2921(1460) ack 1 win 65535 (DF)
14:13:59.889129 IBMSYSTEM.20 > STVMS1.4678: R 2390581621:2390581621(0) win 32768
14:13:59.906706 IBMSYSTEM.21 > STVMS1.3834: P 350:399(49) ack 93 win 65522
14:13:59.915495 STVMS1.3834 > IBMSYSTEM.21: . ack 399 win 65535


There is a clear difference in the wscale between both sides, wscale 0on my side and wscale 1 on the other.


Does one of the VMS cracks know how to change the wscale?
I have tried both:
$ tcpip set protocol tcp /window_scale
and
$ sysconfig -r inet tcp_dont_winscale=0
to get the Window scale: enabled

But with a new tcpdump the wscale is still 0.

Thanks,
Arno

[rick jones]

Well, that the IBM system is not always showing 65535 in the window field of the ACK's it is sending is "normal" - it ise sending ACKs and saying how much space (remember to multiply by two since the scale factor is one) in the window remains. When you see the window field getting smaller like that, it means that data is arriving, but not being consumed by the remote application.

The second half of your trace is more disturbing - it is showing a boatload of TCP retransmissions and no ACK's from the IBM system. That would do some really nasty things to performance and I think I saw a ReSeT segment in there as if one side gave up (IIRC it was the VMS side, probably after retransmit timing-out).

[Arno van Rennes]

Well I have found the solution!

- This is what I have changed (and did not help):

socket:
somaxconn = 1024 to 65535 (recommended by HP)
sominconn = 1 to 65535 (recommended by HP)

inet:
ipport_userreserved = 5000 to 65535 (recommended by HP)
ipport_userreserved_min = 1024 to 49152 (recommended by HP)
tcp_keepidle = 250 to 14400 (default value)
tcp_keepalive_default = 0 to 1 (and back to 0 again)
tcp_recvspace=61440 to 65535 (and back to 61440 again)
tcp_sendspace = 61440 to 65535 (and back to 61440 again)
tcp_ttl = 60 to 128 (and back to 60 again)

-----------------------------------------

* And this is what did the trick:

$ sysconfig -r inet pmtu_enabled=0
(pmtu_enabled was enabled)

HP indicates:
(in http://h71000.www7.hp.com/doc/732final/documentation/pdf/aa-rn1vb-te.pdf)
2.1.6.10 Disabling Use of the PMTU Protocol:
Packets transmitted between servers are fragmented into units of a specific size in order to ease transmission of the data over routers and small-packet networks, such as Ethernet networks. When the inet subsystem attribute pmtu_enabled is enabled set to 1, which is the default behavior), the system determines the largest common path maximum transmission unit (PMTU) value between servers and uses it as the unit size. The system also creates a routing table entry for each client network that attempts to connect to the server.
Performance Benefits and Tradeoffs:
If a server handles traffic among many remote clients, disabling the use of a PMTU can decrease the size of the kernel routing table, which improves server efficiency. However, on a server that handles local traffic and some remote traffic, disabling the use of a PMTU can degrade bandwidth.
When to Tune:
If an Internet server has poor performance and the routing table increases to more than 1000 entries, you should disable the use of PMTU. This is also recommended if you have a server that handles traffic among many remote clients.
Recommended Values:
To disable the use of PMTU protocol, set the value of the pmtu_enabled attribute to 0.



All other FTP sessions keep function the same way as before so I presume that I can leave it this way!


Thanks to all for your input!


Regards,
Arno