Operating System - OpenVMS
1748034 Members
5070 Online
108757 Solutions
New Discussion юеВ

Re: Using SFTP in a batch job

 
SOLVED
Go to solution
David Froble
Occasional Advisor

Using SFTP in a batch job

I'm trying to use SFTP in a batch command file.

$ SFTP "-D 99" -o "batchmode no" fsrbi@209.172.179.22
fayave8A
ls /TEST/TO/CURRENT
exit

Looks simple enough to me. Doesn't work. So I set the SFTP debug mode to 99, and got lots of diagnostics. It appears that even though I specified the batchmode to be "no", and I had also tried it as "yes", SSH is determining that it is a batch mode, and is therefore surpressing the password prompt.

Short excerpt from the log file:

debug: server offers auth methods 'gssapi-with-mic,password,publickey,keyboard-i
nteractive'.

debug: Ssh2AuthPasswdClient/AUTHC-PASSWD.C:261: In Batchmode, so we're not askin
g the user for password.

debug: Ssh2AuthClient/SSHAUTHC.C:385: Method 'password' disabled.

debug: server offers auth methods 'gssapi-with-mic,password,publickey,keyboard-i
nteractive'.

debug: Ssh2Common/SSHCOMMON.C:185: DISCONNECT received: No further authenticatio
n methods available.
11 REPLIES 11
Hoff
Honored Contributor

Re: Using SFTP in a batch job

You will usually want to have batch-mode enabled (via the "-B" switch), and to set up the matching public certificate on the target box.

The sftp and scp commands aren't (per my experience and per the documentation for at least some of the TCP/IP Services releases) particularly stable around trying to embed a password, if they even permit the password at all. (I know TCP/IP Services V5.5 was pretty wonky here, though there might be (is?) a patch for that.)

The certificate path works rather better here, in general.

Using a straight scp command (with a public key on the far end) might be easier than having that sequence of input, too; you can likely specify the target file path in a quoted string. With the certificates, that likely removes the need for that sequence of embedded commands, too.
Steven Schweda
Honored Contributor

Re: Using SFTP in a batch job

> fayave8A

That's supposed to be the password?

In general, SSH, SCP, SFTP, and friends were
designed not to encourage users to store
passwords in files. (And you can probably
replace "encourage" with "permit". "S"
stands for "Secure", not "Sloppy".)

> [...] SSH is determining that it is a batch
> mode, [...]

More generally, that it has no controlling
terminal, therefore no (real) user
interaction is possible.

> [...],keyboard-interactive'.

They don't call it "keyboard-interactive" for
no reason.

> The certificate path works rather better
> here, in general.

Well, public keys, but yes. If you could get
this to work as you apparently wish to, then
I'd expect someone to fix it eventually, so
that it wouldn't work.
Richard Whalen
Honored Contributor

Re: Using SFTP in a batch job

The two above answers are spot on - SFTP won't do password or keyboard-interactive authentication when running as a batch job. It doesn't matter whether or not you tell it to use batch mode as it will figure out that it doesn't have a controlling terminal.

So, you'll have to set up for public key or host based authentication. See http://h71000.www7.hp.com/doc/83final/ba548_90007/ch03s08.html or http://h71000.www7.hp.com/doc/83final/ba548_90007/ch03s07.html
I recommend public key. Do NOT use a pass-phrase when creating your public key - this is the same as supplying a password and won't help for batch mode. (Though I have heard of some users being able to do strange things with pipes, I've always told them "don't tell me if it breaks because I won't try to fix it".)

When you put your public key on the remote system you need to know the specifics of how the implementation of SSH on that system uses to find the public key and what format it is in. The directions for TCP/IP Services work for TCP/IP Services (and MultiNet and TCPware), but won't work for OpenSSH based implementations, which are quite common. Also TCP/IP Services generates keys that follow the RFC and OpenSSH wants a different format. The SSHKEYGEN program on OpenSSH systems can convert between the two formats.
Steven Schweda
Honored Contributor

Re: Using SFTP in a batch job

> When you put your public key on the remote
> system you need to know [...]

A Forum search for keywords like, say,
ssh key format
should find some relevant old threads. If
you do need help, then actual info about the
systems involved (TCPIP SHOW VERSION,
'SSH "-V"', "uname -a", "ssh -V", ...) might
be useful.

> [...] The SSHKEYGEN program on OpenSSH
> systems [...]

Normally known as "ssh-keygen". "man
ssh-keygen" often works on UNIX(-like)
systems.

> fayave8A

Don't forget to change this password, if that
was an actual password. (We also have a user
name and an IP address.)
David Froble
Occasional Advisor

Re: Using SFTP in a batch job

Yes, that's the IP, user, and password. Just testing, and password will be changed, so I didn't mind posting it if it would help anyone get the procedure working. If anybody wants to hack some lawn mower parts, be my guest.

What I'm reading is that what I need to do just won't work. The trading partner isn't playing with a full deck, and apparently doesn't care if my customer has to manually perform the transfers every day. That just isn't acceptable.

It appears that there will not be any cooperation with the trading partner as far as setting up certificates. Already tried that, but I guess I can grovel some more. Don't think it will help.

So, let me reframe my inquiry. Got a trading partner, wants to use SFTP, will not do more than issue IP, user, and password. Any ideas on how to automate the transfer of some files?
Steven Schweda
Honored Contributor

Re: Using SFTP in a batch job

> [...] What I'm reading is that what I need
> to do just won't work. [...]

It certainly won't be as easy as one might
hope. It might be possible to use Kermit
scripting to fake an interactive (-looking)
session with the SFTP client

> Any ideas on how to automate the transfer
> of some files?

It's not entirely clear to me which client
needs to talk to whose server, and I've never
tried this, but if you can puzzle out the
style of the server's SSH software, then it
might be possible to construct your own key
files, and then use SFTP manually, once, to
install your public key file (and any other
related stuff) on that server. You'd need to
get the file ownership (easy?) and
permissions (perhaps less easy) right on the
server, and you would need write access to
your home directory there. It's all easier
if the server owner cooperates, or if you
have full SSH access instead of only SFTP,
but, if you use public keys, then you really
do want to generate your own key files, so
that your private key remains your private
key.
Hoff
Honored Contributor

Re: Using SFTP in a batch job

If they're connecting into your servers, then they'll have to ship you a public key, because that's just the way sftp and scp works here.

If they're not technically inclined, then generate and give them a private key and tell them that it's their secret password (and it is), and tell them where to locate it in their file structure (you'll need to know the type of system and the ssh/sftp/scp tool), and drop the matching public key onto your own server. (I'd probably then delete the private key from the local servers, but that's your call.)

If you're connecting into them, then drop your own public key into the login directory and add the reference into whatever passes for AUTHORIZATION on the remote box, and Bob's your uncle.
Steven Schweda
Honored Contributor
Solution

Re: Using SFTP in a batch job

> [...] If anybody wants to hack some lawn
> mower parts, be my guest. [...]

You could be in luck.

> [...] Remote version: SSH-2.0-5.3.6.18 SSH Tectia Server

Interestingly, the Tectia SSH software seems
to use the same SSH2-style key file format as
the TCPIP SSH software. Better yet, Tectia
seems still to offer HTML documentation for
their products (unlike some vendors I could
name):

http://www.tectia.com/manuals/server-admin/53/index.html
http://www.tectia.com/manuals/server-admin/53/userauth-pk-authfile.html

According to which, all one should need to do
is drop a suitable public key file into the
".ssh2" directory on the server. Which I
did. Then, any old (Stream_LF, ideally, but
fear not) SFTP batch file can be used without
user interaction. For example:

alp $ type ls-lR.bat
pwd
ls -lR
quit

alp $ sftp "-B" ls-lR.bat fsrbi@209.172.179.22
sftp> pwd
/
sftp> ls -lR
.
TO
TO/ARCHIVE
TO/CURRENT
FROM
FROM/ARCHIVE
FROM/CURRENT
TEST
TEST/TO
TEST/TO/ARCHIVE
TEST/TO/ARCHIVE/NRB02081351084.TXT
TEST/TO/ARCHIVE/NRB02161537200.TXT
TEST/TO/ARCHIVE/NRB11231731097.TXT
TEST/TO/ARCHIVE/NRB02161314095.TXT
TEST/TO/CURRENT
TEST/FROM
TEST/FROM/ARCHIVE
TEST/FROM/CURRENT
.ssh2
.ssh2/authorized_keys
.ssh2/authorized_keys/sms_npp_id_dsa_1024_a.pub
RBI.TXT
sftp> quit

alp $

(Look! It's Uncle Bob!)

Feel free to delete my public key file
whenever it's convenient. (What do I care?
It's a _public_ key file.)

Fine print:

You do need to generate a key pair, if you
don't already have one lying around.

I didn't actually try this in a
non-interactive mode, but if no one asks for
or expects a password, then what could go
wrong?
Steven Schweda
Honored Contributor

Re: Using SFTP in a batch job

> [...] what could go wrong?

Apparently nothing. (Nothing obvious,
anyway.) It worked in a batch job, too, as
expected.

alp $ type ls-lR_bat.com
$ proc_dev_dir = f$parse( "A.;", f$environment( "procedure"))- "A.;"
$ set default 'proc_dev_dir'
$ sftp "-B" ls-lR.bat fsrbi@209.172.179.22

alp $ sub ls-lR_bat.com /log = ALP$DKC0:[SMS.ITRC.froble]ls-lR_bat.log
Job ls-lR_bat (queue SYS$BATCH_ALP, entry 763) started on SYS$BATCH_ALP

Job ls-lR_bat (queue SYS$BATCH_ALP, entry 763) completed

alp $ type ls-lR_bat.log
$ Set NoOn
$ VERIFY = F$VERIFY(F$TRNLNM("SYLOGIN_VERIFY"))
sftp> pwd
/
sftp> ls -lR
.
TO
TO/ARCHIVE
TO/CURRENT
FROM
FROM/ARCHIVE
FROM/CURRENT
TEST
TEST/TO
TEST/TO/ARCHIVE
TEST/TO/ARCHIVE/NRB02081351084.TXT
TEST/TO/ARCHIVE/NRB02161537200.TXT
TEST/TO/ARCHIVE/NRB11231731097.TXT
TEST/TO/ARCHIVE/NRB02161314095.TXT
TEST/TO/CURRENT
TEST/FROM
TEST/FROM/ARCHIVE
TEST/FROM/CURRENT
.ssh2
.ssh2/authorized_keys
.ssh2/authorized_keys/sms_npp_id_dsa_1024_a.pub
RBI.TXT
sftp> quit
SMS job terminated at 15-APR-2011 22:19:38.36

Accounting information:
Buffered I/O count: 1186 Peak working set size: 11760
Direct I/O count: 110 Peak virtual size: 186560
Page faults: 930 Mounted volumes: 0
Charged CPU time: 0 00:00:00.22 Elapsed time: 0 00:00:10.75
alp $


(The batch log file suffers from extra CR
characters on line endings of the directory
listing sent from the server, but I've seen
worse pollution.)