Unable to get SFTP working to remote server

[Dennis Piepel]

I am a longtime OpenVMS software developer and have been trying to get SFTP working for one of our customers but have been unsuccessful.

I have read in several forum posts that the SFTP utility in OpenVMS actually expects "Unix" style syntax, but I am not sure how to interpret that.  Normally, if I were trying to PUT a file from the DKA0:[MYDIR.SUB1] directory, I would enter a command like:

sftp> put [MYDIR.SUB1]TESTFILE.TXT

However, some of the forum posts indicate that the directory syntax should read like:

sftp> put \mydir\sub1\testfile.txt

Is that correct?

Do I need to worry about case sensitivity with SFTP commands?

If so, do I need to enclose arguments in quotes to preserve case?

I have also seen references that indicate that any files sent via SFTP on OpenVMS need to have Stream_LF record format rather than the default OpenVMS text format of "Variable length".

Is that true?

Here is a sample script that I tried using the "-v" option:

$ sftp -v "KTE@integration-test.gtnexus.com#18222"
Sftp2/SFTP2.C:5258: CRTL version (SYS$SHARE:DECC$SHARE ident) is: ELF

SshFileCopy/SSHFILECOPY.C:1358: Making local connection.
Ssh2SftpServer/SSHFILEXFERS.C:2145: Received SSH_FXP_INIT
Ssh2SftpServer/SSHFILEXFERS.C:2190: version is 999
Ssh2SftpServer/SSHFILEXFERS.C:2252: Sending SSH_FXP_VERSION with sftp-version@openvms.hp.com as 3
SshFileXferClient/SSHFILEXFERC.C:1440: ssh_file_client_receive_proc: coming in with extension data, OpenVMS host
SshFileXferClient/SSHFILEXFERC.C:1491: vms_plus_sftp_version = 3
SshFileCopy/SSHFILECOPY.C:1297: Connection to local, ready to serve requests.
Sftp2/SFTP2.C:840: Connection ready.
SshReadLine/SSHREADLINE.C:3662: Initializing ReadLine...
SshFileCopy/SSHFILECOPY.C:1369: Connecting to remote host. (host = KTE@integration-test.gtnexus.com#18222, user = NULL, port = NULL)
argv[0] = /sys$system/tcpip$ssh_ssh2
argv[1] = -v
argv[2] = -x
argv[3] = -a
argv[4] = -o
argv[5] = passwordprompt %U@%H's password:
argv[6] = -o
argv[7] = authenticationnotify yes
argv[8] = KTE@integration-test.gtnexus.com#18222
argv[9] = -s
argv[10] = sftp
Sftp2/SFTP2.C:4442: notification: 0
Sftp2/SFTP2.C:4442: notification: 1

debug(28-AUG-2017 15:35:52.42): Ssh2/SSH2.C:1896: CRTL version (SYS$SHARE:DECC$SHR.EXE ident) is ELF
debug(28-AUG-2017 15:35:52.44): SshAppCommon/SSHAPPCOMMON.C:313: Allocating global SshRegex context.
debug(28-AUG-2017 15:35:52.45): SshConfig/SSHCONFIG.C:3461: Metaconfig parsing stopped at line 4.
debug(28-AUG-2017 15:35:52.45): SshConfig/SSHCONFIG.C:887: Setting variable 'VerboseMode' to 'FALSE'.
debug(28-AUG-2017 15:35:52.47): SshConfig/SSHCONFIG.C:3369: Unable to open ssh2/ssh2_config
debug(28-AUG-2017 15:35:52.47): Connecting to integration-test.gtnexus.com, port 18222... (SOCKS not used)
debug(28-AUG-2017 15:35:52.47): Ssh2/SSH2.C:2881: Entering event loop.
debug(28-AUG-2017 15:35:52.54): Ssh2Client/SSHCLIENT.C:1609: Creating transport protocol.
debug(28-AUG-2017 15:35:52.54): SshAuthMethodClient/SSHAUTHMETHODC.C:104: Added "publickey" to usable methods.
debug(28-AUG-2017 15:35:52.54): Ssh2Client/SSHCLIENT.C:1650: Creating userauth protocol.
debug(28-AUG-2017 15:35:52.54): client supports 1 auth methods: 'publickey'
debug(28-AUG-2017 15:35:52.55): SshUnixTcp/SSHUNIXTCP.C:1750: using local hostname ktes01.KNAPHEIDE.COM
debug(28-AUG-2017 15:35:52.55): Ssh2Common/SSHCOMMON.C:541: local ip = 192.168.100.33, local port = 60700
debug(28-AUG-2017 15:35:52.55): Ssh2Common/SSHCOMMON.C:543: remote ip = 198.186.189.242, remote port = 18222
debug(28-AUG-2017 15:35:52.55): SshConnection/SSHCONN.C:2578: Wrapping...
debug(28-AUG-2017 15:35:52.55): SshReadLine/SSHREADLINE.C:3662: Initializing ReadLine...
debug(28-AUG-2017 15:35:52.63): Remote version: SSH-2.0-TradeCard 1.00 [SERVER]
debug(28-AUG-2017 15:35:52.64): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:52.64): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 20 to connection
debug(28-AUG-2017 15:35:52.64): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:52.64): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 30 to connection
debug(28-AUG-2017 15:35:52.69): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=20
debug(28-AUG-2017 15:35:52.69): Ssh2Transport/TRCOMMON.C:2318: lang s to c: `', lang c to s: `'
debug(28-AUG-2017 15:35:52.69): Ssh2Transport/TRCOMMON.C:2383: c_to_s: cipher 3des-cbc, mac hmac-sha1, compression none
debug(28-AUG-2017 15:35:52.69): Ssh2Transport/TRCOMMON.C:2386: s_to_c: cipher 3des-cbc, mac hmac-sha1, compression none
debug(28-AUG-2017 15:35:52.75): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=31
debug(28-AUG-2017 15:35:52.76): Remote host key found from database.
debug(28-AUG-2017 15:35:52.77): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:52.77): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 21 to connection
debug(28-AUG-2017 15:35:52.77): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:52.77): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 5 to connection
debug(28-AUG-2017 15:35:52.77): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=21
debug(28-AUG-2017 15:35:52.88): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=6
debug(28-AUG-2017 15:35:52.88): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:52.88): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 50 to connection
debug(28-AUG-2017 15:35:52.89): Ssh2Common/SSHCOMMON.C:342: Received SSH_CROSS_STARTUP packet from connection protocol.
debug(28-AUG-2017 15:35:52.89): Ssh2Common/SSHCOMMON.C:392: Received SSH_CROSS_ALGORITHMS packet from connection protocol.
debug(28-AUG-2017 15:35:52.95): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=51
debug(28-AUG-2017 15:35:52.95): server offers auth methods 'keyboard-interactive,publickey,password'.
debug(28-AUG-2017 15:35:52.95): Ssh2AuthPubKeyClient/AUTHC-PUBKEY.C:1677: adding keyfile "/DKA0/NIGHT/ssh2/hostkey" to candidates
debug(28-AUG-2017 15:35:52.95): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:52.95): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 50 to connection
debug(28-AUG-2017 15:35:53.02): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=60
debug(28-AUG-2017 15:35:53.02): Constructing and sending signature in publickey authentication.
debug(28-AUG-2017 15:35:53.02): Ssh2AuthPubKeyClient/AUTHC-PUBKEY.C:869: ssh_client_auth_pubkey_send_signature: reading /DKA0/NIGHT/
ssh2/hostkey
debug(28-AUG-2017 15:35:53.04): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:53.04): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 50 to connection
debug(28-AUG-2017 15:35:53.18): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=52
debug(28-AUG-2017 15:35:53.18): Ssh2AuthPubKeyClient/AUTHC-PUBKEY.C:1915: Public key authentication was successful.
debug(28-AUG-2017 15:35:53.18): Ssh2Common/SSHCOMMON.C:310: Received SSH_CROSS_AUTHENTICATED packet from connection protocol.
Sftp2/SFTP2.C:4442: notification: 0ReadLine/SSHREADLINE.C:3728: Uninitializing ReadLine...

Sftp2/SFTP2.C:4459: read char: ASsh2Common/SSHCOMMON.C:852: num_channels now 1
Sftp2/SFTP2.C:4461: read_bytes: 1, buffer len: 1
Sftp2/SFTP2.C:4463: received message:

00000000: 41 AON.C:1113: Sending packet with type 2 to connection
Sftp2/SFTP2.C:4459: read char: U

Sftp2/SFTP2.C:4461: read_bytes: 2, buffer len: 2COMMON.C:1113: Sending packet with type 90 to connection
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 AU
Sftp2/SFTP2.C:4459: read char: T
Sftp2/SFTP2.C:4461: read_bytes: 3, buffer len: 3
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 54 AUT
Sftp2/SFTP2.C:4459: read char: H
Sftp2/SFTP2.C:4461: read_bytes: 4, buffer len: 4
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 AUTH
Sftp2/SFTP2.C:4459: read char: E
Sftp2/SFTP2.C:4461: read_bytes: 5, buffer len: 5
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 45 AUTHE
Sftp2/SFTP2.C:4459: read char: N
Sftp2/SFTP2.C:4461: read_bytes: 6, buffer len: 6
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e AUTHEN
Sftp2/SFTP2.C:4459: read char: T

Sftp2/SFTP2.C:4461: read_bytes: 7, buffer len: 7COMMON.C:2756: >TR packet_type=91

Sftp2/SFTP2.C:4463: received message:nnection/SSHCONN.C:2306: >CXN packet_type=91
00000000: 4155 5448 454e 54 AUTHENT
Sftp2/SFTP2.C:4459: read char: I
Sftp2/SFTP2.C:4461: read_bytes: 8, buffer len: 8
Sftp2/SFTP2.C:4463: received message:

00000000: 4155 5448 454e 5449 AUTHENTI13: Sending packet with type 2 to connection

Sftp2/SFTP2.C:4459: read char: CSsh2Transport/TRCOMMON.C:1113: Sending packet with type 98 to connection
Sftp2/SFTP2.C:4461: read_bytes: 9, buffer len: 9
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 43 AUTHENTIC
Sftp2/SFTP2.C:4459: read char: A
Sftp2/SFTP2.C:4461: read_bytes: 10, buffer len: 10
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 AUTHENTICA
Sftp2/SFTP2.C:4459: read char: T
Sftp2/SFTP2.C:4461: read_bytes: 11, buffer len: 11
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 54 AUTHENTICAT
Sftp2/SFTP2.C:4459: read char: E
Sftp2/SFTP2.C:4461: read_bytes: 12, buffer len: 12
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 5445 AUTHENTICATE
Sftp2/SFTP2.C:4459: read char: D
Sftp2/SFTP2.C:4461: read_bytes: 13, buffer len: 13
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 5445 44 AUTHENTICATED
Sftp2/SFTP2.C:4459: read char:
Sftp2/SFTP2.C:4461: read_bytes: 14, buffer len: 14
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 5445 4420 AUTHENTICATED
Sftp2/SFTP2.C:4459: read char: Y
Sftp2/SFTP2.C:4461: read_bytes: 15, buffer len: 15
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 5445 4420 59 AUTHENTICATED Y
Sftp2/SFTP2.C:4459: read char: E
Sftp2/SFTP2.C:4461: read_bytes: 16, buffer len: 16
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 5445 4420 5945 AUTHENTICATED YE
Sftp2/SFTP2.C:4459: read char: S
Sftp2/SFTP2.C:4461: read_bytes: 17, buffer len: 17
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 5445 4420 5945 AUTHENTICATED YE
00000010: 53 S
Sftp2/SFTP2.C:4459: read char:

Sftp2/SFTP2.C:4461: read_bytes: 18, buffer len: 18
Sftp2/SFTP2.C:4463: received message:
00000000: 4155 5448 454e 5449 4341 5445 4420 5945 AUTHENTICATED YE
00000010: 530a S.
Sftp2/SFTP2.C:4468: buffer: 'AUTHENTICATED YES
'

debug(28-AUG-2017 15:35:53.25): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:53.25): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 94 to connection
debug(28-AUG-2017 15:35:53.29): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=99
debug(28-AUG-2017 15:35:53.29): SshConnection/SSHCONN.C:2306: >CXN packet_type=99
debug(28-AUG-2017 15:35:53.35): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=94
SshFileCopy/SSHFILECOPY.C:1297: Connection to remote host 'KTE@integration-test.gtnexus.com#18222', ready to serve requests.
Sftp2/SFTP2.C:2364: get pwd

debug(28-AUG-2017 15:35:53.35): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:35:53.35): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 94 to connection
debug(28-AUG-2017 15:35:53.40): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=94
sftp> 28-AUG-2017 15:35:53.40): SshConnection/SSHCONN.C:2306: >CXN packet_type=94
sftp>
sftp>
sftp> put "KNAPH_FORDIT_QM_000046302_170810T12514254.X12"

SshFCRecurse/SSHFC_RECURSE.C:410: File is "raw", and it needs to be parsed.
Ssh2SftpServer/SSHFILEXFERS.C:3260: Received SSH_FXP_STAT
Ssh2SftpServer/SSHFILEXFERS.C:3335: Statting file `KNAPH_FORDIT_QM_000046302_170810T12514254.X12'
SshFCTransfer/SSHFC_TRANSFER.C:2480: File list has 2 files.
SshFCTransfer/SSHFC_TRANSFER.C:576: Next source file is /KNAPH_FORDIT_QM_000046302_170810T12514254.X12 .
Ssh2SftpServer/SSHFILEXFERS.C:3260: Received SSH_FXP_STAT
Ssh2SftpServer/SSHFILEXFERS.C:3335: Statting file `KNAPH_FORDIT_QM_000046302_170810T12514254.X12'

debug(28-AUG-2017 15:37:33.05): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:37:33.05): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 94 to connection
debug(28-AUG-2017 15:37:39.13): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=94
debug(28-AUG-2017 15:37:39.13): SshConnection/SSHCONN.C:2306: >CXN packet_type=94
debug(28-AUG-2017 15:37:39.13): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:37:39.13): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 94 to connection
debug(28-AUG-2017 15:37:39.19): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=94
SshFCTransfer/SSHFC_TRANSFER.C:227: Received error `./KNAPH_FORDIT_QM_000046302_170810T12514254.X12 is not a valid file path' (2).

debug(28-AUG-2017 15:37:39.19): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:37:39.19): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 94 to connection
debug(28-AUG-2017 15:37:39.24): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=94
Ssh2SftpServer/SSHFILEXFERS.C:2306: Received SSH_FXP_OPEN306: >CXN packet_type=94
Ssh2SftpServer/SSHFILEXFERS.C:2400: Downloading `KNAPH_FORDIT_QM_000046302_170810T12514254.X12'

debug(28-AUG-2017 15:37:39.25): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 2 to connection
debug(28-AUG-2017 15:37:39.25): Ssh2Transport/TRCOMMON.C:1113: Sending packet with type 94 to connection
debug(28-AUG-2017 15:37:39.30): Ssh2Transport/TRCOMMON.C:2756: >TR packet_type=94
SshFCTransfer/SSHFC_TRANSFER.C:227: Received error `./KNAPH_FORDIT_QM_000046302_170810T12514254.X12 is not a valid file path' (2).
SshReadLine/SSHREADLINE.C:3728: Uninitializing ReadLine...
FATAL: BUILD26$:[TCPIP_V57_BLECO2.SRC.SSH2]SSHFC_TRANSFER.C;1:1849 SshFCTransfer (function name unavailable) Assertion failed: tdat)

%TCPIP-F-SSH_FATAL, non-specific fatal error condition
$

The KNAPH_FORDIT_QM_000046302_170810T12514254.X12 file that I was trying to "put" was in my current directory, so I don't know why it thinks the default directory is not a valid file path.

All this is very cryptic.

Any help would be appreciated.

Dennis

[Steven Schweda]

   First:

      tcpip show version
      sftp "-V"

   And, do we know anything about the software at the other end?

> However, some of the forum posts indicate that the directory syntax
> should read like:
>
> sftp> put \mydir\sub1\testfile.txt

   That would be Windows format.  UNIX format on VMS might look more
like:

      /DKA0/MYDIR/SUB1/TESTFILE.TXT

And, for file specifications, case seldom matters on VMS.

> Do I need to worry about case sensitivity with SFTP commands?

   Trusting no one, I'd run the experiment.  For example:

alp $ sftp "-V"
alp$dkc0:[sys0.syscommon.][sysexe]tcpip$ssh_sftp2.exe: SSH Secure Shell OpenVMS
(V5.5) 3.2.0 on COMPAQ Professional Workstation  - VMS V8.4

alp $ sftp -V
Sftp2/SFTP2.C:5585: CRTL version (SYS$SHARE:DECC$SHARE ident) is: V8.4-00

SshFileCopy/SSHFILECOPY.C:1362: Making local connection.
Ssh2SftpServer/SSHFILEXFERS.C:2333: Received SSH_FXP_INIT
[...]

   Looks different to me.

> If so, do I need to enclose arguments in quotes to preserve case?

   The usual quotation should do what the usual quotation does.  (As
shown above.)

> SshFCTransfer/SSHFC_TRANSFER.C:227: Received error
> `./KNAPH_FORDIT_QM_000046302_170810T12514254.X12 is not a valid file
> path' (2).

   This looks bad.

   I try never to use SFTP, so I know nothing, but when I tried it from
my system to itself, "-v" spewed similar (but different) stuff:

ALP $ sftp -v sms@alp-l
Sftp2/SFTP2.C:5585: CRTL version (SYS$SHARE:DECC$SHARE ident) is: V8.4-00
[...]
sftp> cd itrc
[...]
sftp> put login.com
SshFCTransfer/SSHFC_TRANSFER.C:311: convert_path in:login.com full:login.com raw
:login.com
SshFCRecurse/SSHFC_RECURSE.C:434: File is "raw", and it needs to be parsed.
Ssh2SftpServer/SSHFILEXFERS.C:3481: Received SSH_FXP_STAT
Ssh2SftpServer/SSHFILEXFERS.C:3501: home directory: SYS$SYSROOT:[SYSMGR]
Ssh2SftpServer/SSHFILEXFERS.C:3568: Statting file 'login.com'
SshFCTransfer/SSHFC_TRANSFER.C:2617: File list has 2 files.
SshFCTransfer/SSHFC_TRANSFER.C:663: Next source file is /login.com .
[...]
SshFCTransfer/SSHFC_TRANSFER.C:261: Received error `syserr: no such file or dire
ctory, or no privilege for attempted operation, file: itrc/./login.com' (2).
      [But...]
Ssh2SftpServer/SSHFILEXFERS.C:2512: Received SSH_FXP_OPEN
Ssh2SftpServer/SSHFILEXFERS.C:2607: Open 'login.com' flags read
Ssh2SftpServer/SSHFILEXFERS.C:2608: Downloading 'login.com'
[...]
SshFCTransferCore/SSHFC_TRCORE.C:860: transfer_write_out entered
login.com                         |  6.9kB |   6.9 kB/s | TOC: 00:00:01 | 100%
SshFCTransferCore/SSHFC_TRCORE.C:891: Writer finished.
SshFCTransfer/SSHFC_TRANSFER.C:2096: Destination file attributes not available o
r that file is not a regular file; not changing attributes.
SshFCTransfer/SSHFC_TRANSFER.C:2190: Finished with file itrc/./login.com.
[...]

   So, despite the quirky messages, the file transfer actually seemed to
work.

ALP $ diff login.com home_sms:[sms.itrc]login.com
Number of difference sections found: 0
Number of difference records found: 0

DIFFERENCES /MERGED=1-
    SYS$SYSROOT:[SYSMGR]LOGIN.COM;128-
    ALP$DKC0:[SMS.itrc]login.com;1

   So, knowing what I do, I'd say that that complaint:

SshFCTransfer/SSHFC_TRANSFER.C:227: Received error
 `./KNAPH_FORDIT_QM_000046302_170810T12514254.X12 is not a valid file
 path' (2).

came from the other end, where someone didn't like what it saw.  Have
you tried a file named, say, "a.b"?

> All this is very cryptic.

   It may help to ignore the parts where something simple is shown as
it's formed, one character at a time.

[simplylinuxfaq]

I'm not sure how this works on "OpenVMS", but by default in linux (RHEL or CentOS etc.,), I've seen that the sftp login would land on users home directory unless modified. So, to get upload a file which is not in user home directory, need to specify the file with absolute path and check if that works. 

[Steven Schweda]

> I'm not sure how this works on "OpenVMS", but by default in linux
> (RHEL or CentOS etc.,), I've seen that the sftp login would land on
> users home directory unless modified.

   That's a server decision, not a client decision, and this question
involved a VMS client with an unspecified server.  And this behavior has
been the norm for FTP servers since approximately the beginning of time.

> So, to get upload a file which is not in user home directory, need to
> specify the file with absolute path and check if that works.

   Or, use "cd", as with approximately every [S]FTP server on the
planet.  "cd" to a home-directory subdirectory was shown above; whether
you're allowed to specify some (UNIX-format?) absolute directory depends
on the [S]FTP server.


> [...] UNIX format on VMS might look more like:

   Or, as the program itself suggests:

debug(28-AUG-2017 15:35:52.95): Ssh2AuthPubKeyClient/AUTHC-PUBKEY.C:1677: adding
 keyfile "/DKA0/NIGHT/ssh2/hostkey" to candidates

[Dennis Piepel]

Steven,

Some feedback per your comments:

$
$ tcpip show version

HP TCP/IP Services for OpenVMS Industry Standard 64 Version V5.7 - ECO 2
on an HP Integrity rx2800 i2 (1.33GHz/4.0MB) running OpenVMS V8.4

$ sftp "-V"
ktes01$dka0:[sys0.syscommon.][sysexe]tcpip$ssh_sftp2.exe: SSH Secure Shell OpenVMS (V5.5) 3.2.0 on HP Integrity rx2800 i2 (1.33GHz
- VMS V8.4
$
$

And, I know very little about the software at the other end.  This project is being driven by my customer who in turn is being tasked with implementing SFTP connectivity to send EDI files to their server.

 Also, I had my "slashes" and "backslashes" confused.  The "Unix" syntax does use the forward slash (/).

Regarding case sensitivity, my concern was not about the actual SFTP options (which I am pretty sure are case sensitive and therefore need to be "quoted" to preserve the uppercase option (since option -v and option "-V" might be two unique options.

Rather, my concern was whether the file specification needed to be in quotes to insure that SFTP found the filename using UPPERCASE characters.

Your comment that the following message "looks bad" is exactly why I am wondering why SFTP appears to not like this file path:

> SshFCTransfer/SSHFC_TRANSFER.C:227: Received error
> `./KNAPH_FORDIT_QM_000046302_170810T12514254.X12 is not a valid file
> path' (2).

That filename definitely was in my default directory.

Therefore I can only assume that either the "Unix" style path (i.e. dot then slash) is not being parsed correctly.

So I am puzzled how I can provide SFTP with a file path that it "likes".

Your "Unix" syntax example seems to indicate that I must include the device name (i.e. DKA0) in my Unix style path.

So for a file named A.TXT in my default directory, I tried various SFTP commands like:

sftp> put a.txt

sftp> put "A.TXT"

sftp> put /DKA0/NIGHT/A.TXT

sftp> put "/DKA0/NIGHT/A.TXT"

All of the above failed with errors like:

SshFCTransfer/SSHFC_TRANSFER.C:227: Received error `./a.txt is not a valid file path' (2).

Finally, your example of doing a successfull SFTP file transfer gives rise to several questions:

1. I see "alp" and "ALP" in your prompts.  Does this mean you are running SFTP on an Alphaserver rather than an Integrity server?

2. You did not indicate what your TCPIP version was nor what your SFTP version was either.

3. I don't know anything about the server you did the "put" to. 

If you were transferring to another OpenVMS server then perhaps your environment is enough different than mine to skew what you see vs. what I see.

[Steven Schweda]

> And, I know very little about the software at the other end.

   That's unfortunate, and may need to change.

> Therefore I can only assume that either the "Unix" style path (i.e. dot
> then slash) is not being parsed correctly.
>
> So I am puzzled how I can provide SFTP with a file path that it "likes".

   To me (knowing what I know), that message looks as if it comes from
the (remote, mystery) server, not from the (local, VMS) client.  I would
not be amazed if the client was adding the leading "./" (just to cause
trouble?), and the (mystery, non-UNIX?) server didn't like it.

> 1. I see "alp" and "ALP" in your prompts.  Does this mean you are
> running SFTP on an Alphaserver rather than an Integrity server?

   Yes.  It was handier.

> 2. You did not indicate what your TCPIP version was nor what your SFTP
> version was either.
> version was either.

   Annoying, isn't it?  On the Alpha:

ALP $ tcpip show version

  HP TCP/IP Services for OpenVMS Alpha Version V5.7 - ECO 5
  on a COMPAQ Professional Workstation XP1000 running OpenVMS V8.4    

ALP $ sftp "-V"
alp$dkc0:[sys0.syscommon.][sysexe]tcpip$ssh_sftp2.exe: SSH Secure Shell OpenVMS
(V5.5) 3.2.0 on COMPAQ Professional Workstation  - VMS V8.4

> 3. I don't know anything about the server you did the "put" to.

> [...] from my system to itself, [...]

   So, the same Alpha system.

   I have a recently acquired IA64 system, with what looks like the same
stuff as yours:

ITS $ tcpip show version

  HP TCP/IP Services for OpenVMS Industry Standard 64 Version V5.7 - ECO 2
  on an HP rx2660  (1.59GHz/9.0MB) running OpenVMS V8.4

ITS $ sftp "-V"
 its$dka1:[sys0.syscommon.][sysexe]tcpip$ssh_sftp2.exe: SSH Secure Shell OpenVMS
 (V5.5) 3.2.0 on HP rx2660  (1.59GHz/9.0MB) - VMS V8.4

   Interestingly, as user SYSTEM, I get little beside trouble from the
SFTP server on that system -- mostly permissions complaints, like:

ALP $ sftp its

 Welcome to HP OpenVMS Industry Standard 64 Operating System, Version V8.4    
sftp> cd sms
SYS$SYSROOT:[SYSMGR.SMS]
sftp> put login.com
open: sms/./login.com (dst): unspecified failure (server msg: 'syserr: bad file
number, or no privilege for attempted operation, file: sms/./login.com')

   As the client (with the Alpha as the server), it does better:

ITS $ sftp alp-l

@ SYS$MANAGER:ANNOUNCE.TXT      [That's a known-cause annoyance.]
sftp> cd sms
alp$dkc0:[SYS0.SYSMGR.SMS]
sftp> put login.com
login.com                         |  6.9kB |   6.9 kB/s | TOC: 00:00:01 | 100%


> If you were transferring to another OpenVMS server then perhaps your
> environment is enough different than mine to skew what you see vs. what
> I see.

   Doubtless.  If I get a chance, I'll try to run a test to a non-VMS
server.  It'll need to be something other than a normally configured
Mac, however, because of the retarded SSH software on these things:

ALP $ sftp sms@pro3
FATAL: ssh2 client failed to authenticate. (or you have too old ssh2 installed,
check with ssh2 "-V")
warning: Authentication failed.

%TCPIP-F-SSH_FATAL, non-specific fatal error condition


   Speaking of retarded, I see that the TCPIP kit on my Alpha says "V5.7
- ECO 5", whereas, on the IA64, it's "V5.7 - ECO 2".  As a lowly
Hobbyist, my patch access is poor, so I know nothing, but if there's a
more advanced TCPIP ECO on IA64, then you might give that a try.  The
TCPIP57ECO05.RELEASE_NOTES content on Alpha is pretty scary regarding
SFTP, but it wasn't clear (to me) that the ECO numbers for Alpha and
IA64 are comparable, nor how much (if any) of the scary stuff came after
ECO 2, but newer is bound to be better, right?

[Steven Schweda]

> [...]  If I get a chance, I'll try to run a test to a non-VMS
> server.  [...]

   Using a handy HP-UX server:

rux$ uname -a
HP-UX rux B.11.31 U ia64 1678555272 unlimited-user license

ITS $ sftp -v sms@rux
  Sftp2/SFTP2.C:5258: CRTL version (SYS$SHARE:DECC$SHARE ident) is: ELF
      [Hmmm.  On Alpha, that "CRTL version" is more informative.]
[...]
sftp> cd itrc
[...]
sftp> put login.com
[...]
SshFCTransferCore/SSHFC_TRCORE.C:369: Starting transfer for file login.com, dest
ination itrc/./login.com
[...]
SshFCTransferCore/SSHFC_TRCORE.C:846: transfer_write_out entered
login.com                         |  6.6kB |   6.6 kB/s | TOC: 00:00:01 | 100%
SshFCTransferCore/SSHFC_TRCORE.C:877: Writer finished.
SshFCTransfer/SSHFC_TRANSFER.C:1960: Destination file attributes not available o
r that file is not a regular file; not changing attributes.
SshFCTransfer/SSHFC_TRANSFER.C:2053: Finished with file itrc/./login.com.
[...]

   Again with the unmotivated "./" in the path.  I think that we can
blame the (VMS/TCPIP) client for this quirk.  With a UNIX system like
this HP-UX system as a server, "itrc/./login.com" is equivalent to
"itrc/login.com", and the transfer works with no problems.  If, however,
you're dealing with an IBM z/OS server (or some such non-UNIX system),
then a "./" in the path might cause trouble, possibly of the "is not a
valid file path" variety.  I didn't see a way to stop the SFTP client
from adding that "./" to the destination path.

   How wedded are you to SFTP?  A quick test using (what should be) an
equivalent SCP command seems to show no such fiddling with the
destination path:

ITS $ scp -v login.com sms@rux:itrc/login.com
  tcpip$ssh_scp2.exe:Scp2/SCP2.C:1993: CRTL version (SYS$SHARE:DECC$SHARE ident)
 is: ELF
[...]
tcpip$ssh_scp2.exe:SshFCTransferCore/SSHFC_TRCORE.C:369: Starting transfer for f
ile login.com, destination itrc/login.com
[...]
tcpip$ssh_scp2.exe:SshFCTransfer/SSHFC_TRANSFER.C:2053: Finished with file itrc/
login.com.
[...]

   That was with an explicit destination directory, however.  It does
seem to mutilate a bare file name in the same old way, so this may be
less of an improvement that I had first thought:

ITS $ scp -v login.com sms@rux:              
[...]
tcpip$ssh_scp2.exe:SshFCTransferCore/SSHFC_TRCORE.C:369: Starting transfer for f
ile login.com, destination ./login.com
[...]

   Ever helpful, this stuff.  So, if the leading "./" is actually the
source of the problem, and if you can't use SCP and specify some other
destination directory, then I see no hope, other than getting this
software straightened out.  I wouldn't care to guess how easy that would
be.

[Dennis Piepel]

Thanks Steven for your advice,

After much testing it appears to me that OpenVMS is not a viable platform for SFTP file transfers for my customer's application. 

For whatever reason, the SFTP process appears to expect filenames and directory syntax to be "Unix" format and I believe that OpenVMS is unable to parse that syntax in such a way as to locate the document to transfer.

Therefore, I will be recommending to my customer that they use some protocol other than SFTP for this project.

FYI, the company that has the remote server is GT Nexus and they also offer AS2 as an EDI file transfer method.

I know nothing about AS2 and have no idea if that is a viable protocol for file transfers to/from an OpenVMS server.

Any chance you know anything about programming an AS2 file transfer?

Regardless, thanks giving me some feedback on this!

[Steven Schweda]

> After much testing it appears to me that OpenVMS is not a viable
> platform for SFTP file transfers for my customer's application.

   Perhaps.  I'd check for a newer TCPIP ECO before condemning the whole
lot, however.  A formal complaint to HPE couldn't hurt, either.

> For whatever reason, the SFTP process appears to expect filenames and
>directory syntax to be "Unix" format and I believe that OpenVMS is
>unable to parse that syntax in such a way as to locate the document to
>transfer.

   I think that the problem is different.  As I read it, the VMS client
can find the local file well enough, but it forms a UNIX-like
destination path ("./whatever" instead of (plain) "whatever"), and the
destination server (whatever that might be) can't cope with that
unexpected, unnecessary, undesired "./" which was inserted by the VMS
client.

   So, the problem (as I see it) is that the VMS client forms a
too-complex UNIX-like destination path, and the destination server hates
it.  I'd still blame the VMS client, but, as the test with an HP-UX
server shows, a more competent server could work around the apparent
defect in the VMS client.

> FYI, the company that has the remote server is GT Nexus and they also
> offer AS2 as an EDI file transfer method.
>
> I know nothing about AS2 [...]

   I can beat that.  I know nothing about both GT Nexus and AS2.  I may
try to do some reading if/when I get bored.

[Volker Halle]

Dennis,

please consider to install a more recent ECO than TCPIP V5.7 ECO 2.

There are a couple of SFTP fixes in ECO 3 - one of them might look related:

28-July-2011    Integrity servers and Alpha

        Problem:

        SFTP/SCP PUT command to an SSH maverick server fails because of
        the following assertion failure:

        SshFCTransfer (function name unavailable) Assertion failed:
        tdata->current_dest_file->attributes !=
        ((void *) 0)

We had a customer project with various SFTP servers (Redhat Linux and a MOVEit DMZ server) and had no problems copying files from the TCPIP V5.7 ECO 3 OpenVMS SFTP clients to those servers.

And there are even more recent ECOs than TCPIP V5.7 ECO 3 - please log a call with HPE to obtain those.

Volker.

[Steven Schweda]

> We had a customer project with various SFTP servers (Redhat Linux and
> a MOVEit DMZ server) and had no problems copying files from the TCPIP
> V5.7 ECO 3 OpenVMS SFTP clients to those servers.

   How "various"?  UNIX(-like) servers seem not to mind the extra "./"
which the TCPIP client(s) seem to be adding to the destination paths.
The challenge is to find a non-UNIX(-like) server which hates it as much
as the one in this report.

[john Dite]

Dennis,

this is what you should have.

Secure Shell:
TCPIP$SSH_SCP2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]
TCPIP$SSH_SFTP-SERVER2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]
TCPIP$SSH_SFTP2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]
TCPIP$SSH_SSH-ADD2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]
TCPIP$SSH_SSH-AGENT2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]
TCPIP$SSH_SSH-KEYGEN2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]
TCPIP$SSH_SSH-SIGNER2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]
TCPIP$SSH_SSH2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]
TCPIP$SSH_SSHD2;2 "V5.7-ECO5G" 26-NOV-2015 SYS$COMMON:[SYSEXE]

In case you're interested in AS2 we also have an offering that is available on OpenVMS.

John

[Mike Kier]

We've been stuck at older versions of VMS on Alpha and VAX so we went with the Process Multinet SSH/SCP2/SFTP2 add in to TCP/IP Services.

We use both SCP and SFTP extensively as we have shut down DECnet and are in the process of disabling ftp, telnet, the "r" commands and all other insecure protocols under a mandate from our security folks (we handle a lot of PII and PCI data).  We also do not use any ODS5 volumes, so from the VMS side all filenames are uppercase.

SFTP works well for most of our transfers (mostly to/from other VMS systems, RHEL systems, and a few Solaris systems) - we use it mostly in batch and take care to use the LCD and CD commands to set the default locations for both sides of the link.  SFTP doesn't allow us to specifiy the filename on the remote side, however, so for some of our transfers we use SCP which is a bit more restrictive, but does allow us to fully specify the Unix-format mixed case and symbols file names and the input and output filenames can be different.

We had a pretty sizable learning curve  (still ongoing) and there are a lot of Logical Names that Process Multinet uses the can have a big impact on how things are performed, but overall we've been able to accomplish just about everything we set out to do.  You do have to keep in mind that SFTP is NOT regular old FTP with security - it has many differences from FTP.

[Dennis Piepel]

Thanks, but I don't think the Process Multinet add-in would be an option at my customer's site.  Our customers tend to stay with standard, native OpenVMS applications.

[Dennis Piepel]

I talked with HP Support and was able to download the Integrity OpenVMS TCPIP Services v5.7 ECO5 including the ECO 5J patches.

We will see if that makes any difference.

[Dennis Piepel]

HP Support helped me download the TCPIP Services ECO 5 including the ECO5J patches.  I will try those and see if it helps.

[john Dite]

Dennis,

TCPIP Services ECO 5 including the ECO5J patches ?
Not heard of Version ECO5J before can you do a
$ pipe tcpip sho vers/all | sea sys$pipe ssh
and post the result.

Please look in the release notes and give us a clue what ECO5J included

Many thanks.
John

[Dennis Piepel]

John,

The so-called ECO5J patch kit was just an OpenVMS backup set provided to me by HP Technical Support.

The backup set name was "qxcm1001500217_eco5j_ia64.bck"

Here are the installation instructions that I received for this remedial patch:

Remedial fix:

IA64 Saveset:
 QXCM1001500217_1001500217_2016-08-30.BCK;1

Image Identifier:
 V5.7-ECO5J

Link Date:
 26-AUG-2016

Files (IA64):
 TCPIP$SSH_SCP2.EXE;1
 TCPIP$SSH_SFTP-SERVER2.EXE;1
 TCPIP$SSH_SFTP2.EXE;1
 TCPIP$SSH_SSH-ADD2.EXE;1
 TCPIP$SSH_SSH-AGENT2.EXE;1
 TCPIP$SSH_SSH-KEYGEN2.EXE;1
 TCPIP$SSH_SSH-SIGNER2.EXE;1
 TCPIP$SSH_SSH2.EXE;1
 TCPIP$SSH_SSHD2.EXE;1

Installation Instructions:

(1) On target system, stop SSH client and server using shutdown scripts:

$ @sys$STARTUP:TCPIP$SSH_CLIENT_SHUTDOWN.COM
$ @sys$STARTUP:TCPIP$SSH_SHUTDOWN.COM

(2) Copy the images  to respective locations:

$ COPY TCPIP$SSH_SCP2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SCP2.EXE;0
$ COPY TCPIP$SSH_SFTP-SERVER2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SFTP-SERVER2.EXE;0
$ COPY TCPIP$SSH_SFTP2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SFTP2.EXE;0
$ COPY TCPIP$SSH_SSH-ADD2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SSH-ADD2.EXE;0
$ COPY TCPIP$SSH_SSH-AGENT2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SSH-AGENT2.EXE;0
$ COPY TCPIP$SSH_SSH-KEYGEN2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SSH-KEYGEN2.EXE;0
$ COPY TCPIP$SSH_SSH-SIGNER2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SSH-SIGNER2.EXE;0
$ COPY TCPIP$SSH_SSH2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SSH2.EXE;0
$ COPY TCPIP$SSH_SSHD2.EXE SYS$COMMON:[SYSEXE]TCPIP$SSH_SSHD2.EXE;0

(3) Start SSH client and server:

$ @sys$STARTUP:TCPIP$SSH_CLIENT_STARTUP.COM
$ @sys$STARTUP:TCPIP$SSH_STARTUP.COM

Here is the results of the pipe you wanted:

$ pipe tcpip sho vers/all | sea sys$pipe ssh
  tcpip$ssh_scp2;1          "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
  tcpip$ssh_sftp-server2;1  "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
  tcpip$ssh_sftp2;1         "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
  tcpip$ssh_ssh-add2;1      "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
  tcpip$ssh_ssh-agent2;1    "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
  tcpip$ssh_ssh-keygen2;1   "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
  tcpip$ssh_ssh-signer2;1   "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
  tcpip$ssh_ssh2;1          "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
  tcpip$ssh_sshd2;1         "V5.7-ECO5J"      26-AUG-2016  SYS$SYSROOT:[SYSEXE
$

[Dennis Piepel]

We installed the TCPIP Services ECO5 patches last night including the ECO5J remedial patch.

After installing the patches we re-ran our test file transfer and our contact at the remote server indicated that the SFTP file transfer was successfully received.

So it looks like the patches worked!

[john Dite]

Dennis,

thanks for the update. I realise now that it was a Backup kit with installation instructions. I've asked my friendly HPE support to inquire what issues this kit resolved and whether this included any new ciphers etc. compared to ECO5G.

John