Catching the SFTP output in a file

[john nicoara]

Does someone have experience with the re-direction of the SFTP output to a file? I'm burden with the SFTP extricancies using eg. sys$output for 'ls' (list dir) but sys$error for 'get'! Re-directing both sys$output and sys$error to the same file ones get different versions of the same filename!

john

[Richard Whalen]

What is your intended use for the output?

[John Gillings · Accepted solution]

john,

I've found capturing output and status from SSH and SFTP is a bit tricky. They seem to have some kind of assumption about the output device, based on what they see as SYS$COMMAND. Hence the message you will sometimes see as the first output.

"Warning: Need basic cursor movement capability, using vt100"

Redirecting SYS$COMMAND to NL: seems to help.

Here's a possible solution using the PIPE command.

$ PIPE (DEFINE/USER SYS$COMMAND NL: ; -
SFTP "-B" commands "user@node" ; -
DEFINE/JOB/NOLOG SFTP_STAT &$STATUS) | -
COPY SYS$PIPE: SFTP.OUT

At completion, the job logical name SFTP_STAT will contain the completion status of the SFTP command, and SFTP.OUT will contain all the output from the command. I tested this with "ls" and "get" commands. The output from both was written to a single version of SFTP.OUT

[John Gillings]

forgot to mention...

depending on what you want to do with your output, you might not need to write it to a file at all. You can obviously replace the COPY command in that pipeline with SEARCH or any other command, program or procedure you want.

[Thomas Ritter]

From a working example ...
$ fname = "sftp.lis"
$ open/write out sftp_dir.tmp
$ write out "cd ''csa_output_directory'"
$ write out "ls -l"
$ write out "quit"
$ close out
$ convert/fdl=sys$input: sftp_dir.tmp sftp_dir.cmd
RECORD; FORMAT STREAM_LF
$ delete sftp_dir.tmp;
$ type sftp_dir.cmd
$!
$MAIN:
$
$ define sys$output 'fname'
$ sftp "-B" sftp_dir.cmd -o "quietmode yes" secret@192.168.0.1
$ deassign sys$output

[John Gillings]

Thomas,

small improvement... Rather than create a file then CONVERT it, you can create it with the correct record format, like this

$ CREATE/FDL=sys$input: sftp_dir.cmd
RECORD; FORMAT STREAM_LF
$ OPEN/APPEND out sftp_dir.cmd
$ write out "cd ''csa_output_directory'"
$ write out "ls -l"
$ write out "quit"
$ close out

(just doing my bit for the elimination of unnecessary temporary files ;-)

On V8.3 and above, the CREATE can be written as:

$ CREATE/FDL="RECORD; FORMAT STREAM_LF" sftp_dir.cmd

Also note that your procedure only works in batch, as sftp is sensitive to SYS$COMMAND. If executed interactively, the output is sent to the terminal, and an empty file is created. Further, it doesn't get around john's original problem - output from ls goes to SYS$OUTPUT and output from get goes to SYS$ERROR. If you add a GET command to your batch file, the GET output goes to the terminal and generates an empty output file (possibly obscuring the ls output).

The PIPE command, with the critically important "DEFINE/USER SYS$COMMAND NL:" will consolidate all output, and will behave the same as INTERACTIVE or BATCH.

(I have no idea why SSH and SFTP behave so strangely with their output streams)

[Thomas Ritter]

Thanks John, here is the get extract. Works under ucx 5.4 eoc5 but not eco6. go figure :(

Notice the user of define/user sys$error nl:


$ open/write out sftp_bss_get.tmp
$ write out "cd ''csa_output_directory'"
$ write out "lcd DIR:[INPUT_TMP]"
$ write out "get ''sftp_get_file_name'"
$ write out "quit"
$ close out
$!
$ convert/fdl=sys$input: sftp_bss_get.tmp sftp_bss_get.cmd
RECORD; FORMAT STREAM_LF
$!
$ type sftp_bss_get.cmd
$!
$ delete sftp_bss_get.tmp;
$ define/user sys$error nl:
$ sftp "-B" sftp_bss_get.cmd -o "quietmode yes" secret@192.168.0.1

[john nicoara]

Hi John, hi Thomas

Let me express my gratitude for having worried about my problem. Unfortunately it was just before I leaved for the Christmas/New Year holidays, that's why my such late reply. Now, before re-entering the theme, let me wish you all

A Happy New Year and all the best!:)

Back to my SFTP problem, my actual "entry point" is a Perl script from which I have to open a SSH session with a (unix) server, transfer given files and delete them afterwards. As I could not find any Perl library SFTP module running on VMS, I had to deviate via the HP SFTP tool with its known intricacies.
I spent this morning testing your both variants. John's one works fine for me, Thomas' one, at least in my context, not.
Based on John's solution which embeds the call of the SFTP in a single PIPEd command I could, in turn, embed that command in a Perl system() call, as the excerpt below shows:

my $fn_cmd = 'sftpcmd.txt'; #contains SFTP commands, eg. an 'ls' and more 'get's
my $flags = "\"-B\" $fn_cmd";
my $sftpstatus = "$host" . "_sftp_status";
my $fn_eo = 'sftp2vms_test_syssftp.log'; #SFTP output file
#
#build the system() call arguments:
my $sysarg = "PIPE (DEFINE/USER SYS\$COMMAND NL: ;SFTP $flags $user\@$host ;";
$sysarg = $sysarg . "DEFINE/JOB/NOLOG $sftpstatus \&\$STATUS) | COPY SYS\$PIPE: $fn_eo";
print "...\'System\' command is: $sysarg\n";

if ((my $call_stat=system($sysarg)) <0) {
die "VMS PIPE/SFTP execution error, ret.code=$call_stat." ;
}
print "VMS PIPE/SFTP returned, ret.code=$call_stat.\n";

On return from system() call sftp2vms_test_syssftp.log contains both the ls and get results. The host-specific logical <$host>_sftp_status allowing the evaluation of the SFTP execution could be retrieved by means of an existing VMS Perl module (I have to try it first).

The Thomas' variant unfortunately does not work (under TCPIP V5.6-9ECO2 we currently have): the ls' output is correctly re-directed into a file, but the get's output is completely missing (neither in file nor on terminal)... Perhaps do I need some code more as in your excerpts, don't I Thomas? Or you have to add our TCPIP version to your exceptions list, haven't you?

Whatever my test results are, you were both very helpful to me, so once again many thanks for your efforts for giving the HP forum an active life existence.

[Dan Letobar]

I'm having the exact same problem. I need to capture the output from the SFTP ls command to a file so I can search it.

When I do things the normal way:
$assign sys$output sftp.out
$sftp "-b" commands.dat user@node
$deassign sys$output
I get "Warning: Need basic cursor movement capablity, using vt100", and the output goes to the screen.

As soon as I use DEFINE/USER SYS$COMMAND NL:, either in the pipe command or issued separately before the sftp command, I get "FATAL: OpenVMS$tty_gettty() failed
%TCPIP-F-SSH_FATAL, non-specific fatal error condition"

Any idea why I'm getting the error, or any other way to redirect the output to a file?

Thanks.

[Steven Schweda]

> I'm having the exact same problem. [...]

So, refer to the old, dead thread when you
start your own, new thread.

Does adding '-o "batchmode yes"' to the
command do anything for you?

[john nicoara]

Hi Dan,
see the John Gillings' solution above (Dec 18, 2007). It's tricky, but it works perfectly!