topic Re: SFTP and large files in Operating System - OpenVMS

SFTP and large files

Allan Large — Mon, 27 Jul 2009 14:36:47 GMT

We have a site running OpenVMS V8.3-1h1, TCPIP 5.6 ECO 4 that is trying to SFTP files from a non-VMS site. When trying to get some of the large files, SFTP terminates with an error of "packet too long".

I can't find any restrictions on transferring large files. The only thing I found in the release notes said:

3.10.14 SSH Transferring Large Files

This section includes restrictions pertaining to transferring large files:

* The minimum version of DECC$SHR running on your system must be that which was released with OpenVMS Version 8.2.
* You may need to adjust memory parameters (WSDEF, WSQUO, WSEXTENT, and PGFLQUO) to accommodate the memory requirements of the file copy client and server. The exact value depends on system resources and virtual memory configuration. For more information, see Section 2.3. For ssh filecopy, testing has shown that the main parameter to adjust is PGFLQUO.

Another strange thing is that if you try the GET multiple times, the number reported in the "packet too long" message varies.

Here is output from one such GET:

sftp> ls -l
drwxrw-rw- 1 user group 0 Jul 13 10:44 archive
-rw-rw-rw- 1 user group 7942 Jul 25 00:54 dly0967
-rw-rw-rw- 1 user group 4332 Apr 28 00:52 dly0967.04272009
-rw-rw-rw- 1 user group 22382 May 2 01:07 dly0967.05012009
-rw-rw-rw- 1 user group 4332 May 6 00:55 dly0967.05052009
-rw-rw-rw- 1 user group 1083 May 8 00:54 dly0967.05072009
-rw-rw-rw- 1 user group 4332 May 14 00:54 dly0967.05132009
-rw-rw-rw- 1 user group 1444 May 15 00:58 dly0967.05142009
-rw-rw-rw- 1 user group 1444 May 19 00:54 dly0967.05182009
-rw-rw-rw- 1 user group 34295 Jun 2 01:30 dly0967.06012009
-rw-rw-rw- 1 user group 4332 Jun 3 00:59 dly0967.06022009
-rw-rw-rw- 1 user group 1083 Jun 4 00:58 dly0967.06032009
-rw-rw-rw- 1 user group 4332 Jun 10 00:54 dly0967.06092009
-rw-rw-rw- 1 user group 4693 Jun 19 00:55 dly0967.06182009
-rw-rw-rw- 1 user group 4332 Jul 2 01:15 dly0967.07012009
-rw-rw-rw- 1 user group 1083 Jul 10 01:03 dly0967.07092009
-rw-rw-rw- 1 user group 7942 Jul 11 00:56 dly0967.07102009
-rw-rw-rw- 1 user group 4332 Jul 15 00:58 dly0967.07142009
-rw-rw-rw- 1 user group 3971 Jul 16 00:58 dly0967.07152009
-rw-rw-rw- 1 user group 1083 Jul 21 00:56 dly0967.07202009
-rw-rw-rw- 1 user group 7942 Jul 24 00:55 dly0967.07232009
-rw-rw-rw- 1 user group 7942 Jul 25 00:54 dly0967.07242009
-rw-rw-rw- 1 user group 0 Jul 6 13:10 ERS
-rw-rw-rw- 1 user group 93670 Jun 3 19:18 findtl0967.01012009
-rw-rw-rw- 1 user group 94010 May 15 14:55 findtl0967.04012009
-rw-rw-rw- 1 user group 93840 Jun 15 16:53 findtl0967.05012009
-rw-rw-rw- 1 user group 94180 Jul 13 13:52 findtl0967.06012009
-rw-rw-rw- 1 user group 92480 Jun 3 19:14 findtl0967.09012008
-rw-rw-rw- 1 user group 92480 Jun 3 19:15 findtl0967.10012008
-rw-rw-rw- 1 user group 92990 Jun 3 19:16 findtl0967.11012008
-rw-rw-rw- 1 user group 92820 Jun 3 19:17 findtl0967.12012008
-rw-rw-rw- 1 user group 1328119 May 10 11:50 mth0967.04302009
-rw-rw-rw- 1 user group 1325592 Jun 9 16:36 mth0967.05312009
-rw-rw-rw- 1 user group 1324509 Jul 9 18:23 mth0967.06302009
-rw-rw-rw- 1 user group 87252 Jul 9 16:51 mthretfml0967
-rw-rw-rw- 1 user group 86592 May 10 06:09 mthretfml0967.04302009
-rw-rw-rw- 1 user group 86592 Jun 9 14:54 mthretfml0967.05312009
-rw-rw-rw- 1 user group 87252 Jul 9 16:51 mthretfml0967.06302009
-rw-rw-rw- 1 user group 492 Jul 13 10:43 o967457.old
-rw-rw-rw- 1 user group 106 Jul 1 10:32 o967dcp

sftp> get mth0967.06302009

Disconnected; protocol error (Protocol error: packet too long: 64372.).

Warning: child process (/sys$system/tcpip$ssh_ssh2) exited with code 131.

%TCPIP-E-SSH_FC_ERROR, error in ssh file transfer operation

Any suggestions ?

Re: SFTP and large files

Allan Large — Mon, 27 Jul 2009 14:39:33 GMT

Forgot to add that we can get the files that are 94,000 bytes just fine.

Re: SFTP and large files

Wim Van den Wyngaert — Mon, 27 Jul 2009 14:56:53 GMT

Found this (not the samer but comparable):
I've successfully tested TCPIP V5.4 ECO7J and V5.6 ECO2M ssh images and ls no longer results in "packet too long" and gets a successful listing.

Wim

Re: SFTP and large files

Allan Large — Mon, 27 Jul 2009 15:11:08 GMT

And do you know how would one go about obtaining the V5.6 ECO2M images ?

Re: SFTP and large files

Steven Schweda — Mon, 27 Jul 2009 15:36:56 GMT

> And do you know how would one go about
> obtaining the V5.6 ECO2M images ?

Perhaps:
ftp://ftp.itrc.hp.com/openvms_patches/layered_products/i64/
?

I don't know about the "J" and "M". but there
seems to be "HP TCP/IP Services for OpenVMS
V5.6 Update ECO 4" there.

Re: SFTP and large files

Hoff — Mon, 27 Jul 2009 15:43:04 GMT

If it's not posted, call HP.

Re: SFTP and large files

James T Horn — Tue, 28 Jul 2009 12:27:07 GMT

Not sure if it will be the same, but we had the same issue. Currently:
Openvms 8.3 Alpha
TCPIP V5.6 - ECO 3

What we do is convert the file to Stream_LF and that fixed the problem.

Contents of strem_lf.fdl:
SYSTEM
SOURCE OpenVMS

FILE
ORGANIZATION SEQUENTIAL

RECORD
BLOCK_SPAN yes
CARRIAGE_CONTROL carriage_return
FORMAT stream_LF
SIZE 0

Re: SFTP and large files

Richard W Hunt — Wed, 05 Aug 2009 14:23:44 GMT

Packet too long? I would interpret this as a statement that you have mismatched network settings on sender vs. receiever. In LANCP, you should be able to show the device characteristics as

$ MC LANCP SHOW DEVICE/CHAR devname

The questions I would ask include whether the device buffer sizes match (but these are negotiated so SHOULDN'T give trouble) and whether the partner node has jumbo frames enabled.

Re: SFTP and large files

Allan Large — Wed, 05 Aug 2009 14:28:57 GMT

Thanks everyone for the suggestions.

What we found was that the server side of the connection is Globalscape's FTP server. This particular version didn't honor the packet size requested from VMS and thus an error occurred.

http://forums.globalscape.com/tm.aspx?m=15249&mpage=1&key=packet%2ctoo%2clong㮘