SFTP in batch mode - Re: thread 985928

[Erik Harclerode]

Hi,
My apologies in advance for the long thread but I tried to be thorough and answer as many questions up front. I've done lots of searching on these forums and it seems that the definitive thread for SFTP in batch mode is this thread:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=985928

After looking it over I still have a few questions.

First, my situation is that I'm trying to FTP from OpenVMS to an external Unix server. I, the client, am set up to accept all forms of authentication. The server is set up to accept publickey and password authentication. I am able to connect successfully manually, but not through batch mode. When I do connect manually it asks for 2 passwords: first the key password and then the account password.

I've created a file named batchtest.txt located in a drive named DSA106:[programmer]. I've also set up my IDENTIFICATION. file to point to the proper key as well as the SSH2_CONFIG. file to be set up with the host I'm trying to connect to, I'll call it "testhost" in this example.

I try to connect as follows at the $ prompt:
SFTP "-B" DSA106:[programmer]batchtest.txt testhost

batchtest.txt looks as follows:
"""[lf]

where [lf} is the ascii line feed command (in editor I used CTRL-V then CTRL-J)

After executing my SFTP command I receive this error:
Authentication failed.
Disconnected; authentication cancelled by user (Authentication cancelled by user
.).
FATAL: ssh2 client failed to authenticate. (or you have too old ssh2 installed,
check with ssh2 "-V")

%TCPIP-F-SSH_FATAL, non-specific fatal error condition


Here is further information from -v:
debug: server offers auth methods 'publickey,password'.
debug: Ssh2AuthPubKeyClient/AUTHC-PUBKEY.C:1682: adding keyfile "/DSA106/program
mer/ssh2/ID_DSA_2048_A" to candidates
debug: Constructing and sending signature in publickey authentication.
debug: Ssh2AuthPubKeyClient/AUTHC-PUBKEY.C:874: ssh_client_auth_pubkey_send_sign
ature: reading /DSA106/programmer/ssh2/ID_DSA_2048_A
debug: Ssh2Common/SSHCOMMON.C:185: DISCONNECT received: Authentication cancelled
by user.


I've tried everything I could find online and have had no success, so I was hoping someone else could shed some light on this situation for me. What am I doing wrong?

I also notice in the 985928 thread that he uses something called login.com. I have a login.com on the system but there is nothing in it, so I don't see how it would add value to my batch mode.

Any help would be appreciated.

[Volker Halle]

Erik,

I have been able to set up hostbased authentication based on the SSH manual pointed to in the other thread more than 2 years ago. I've never used it again since, so I would have to start all over now...

Maybe the fundamental misunderstanding is, that you can't use passwords stored in files with SFTP in batch mode. You need to configure hostbased authentication and this needs to be done both on the SSH client and SSH server.

Volker.

[Mårten Holmberg]

Hi,
we uses SFTP in batch with publickey and not hostbased authentication.

This is how we got it to work.

$ ! Load key-pair to memory
$ ! -----------------------------------
$ SET NOVER ! Hide passwords from log
$ @sys$manager:tcpip$define_commands
$ open/write add_file temp.file
$ write add_file "$ssh_add -p id_dsa_ssh2"
$ write add_file "''pwd'"
$ close add_file
$
$ ssh_agent
$ wait 00:00:10 ! Wait for SSH_AGENT to start
$ @temp.file
$ delete/log temp.file;*
$ REST_VERIFY ! Restore Verify state
$ ssh_add -l ! List the key we loaded
$
... create sftp-batchfile...
... convert it to streamLF...
$
$ sftp "-B" sftp-batch.file "''usr'"@'host'
$

We get have a password database that we call to get the symbols HOST, USR and PWD and the symbol REST_VERIFY is setup to restore the verify state to what it was before we used the password.

Try a test batch-file with something like:
ls[lf]
exit[lf]
in it at first.

I'm not sure that the wait for the SSH_AGENT is needed, but when we exerimented to get this to work we sometimes got an agent error.

Hope this helps

/MÃ¥rten

[Steven Schweda]

In any situation where you would need to
store a password/passphrase in a file, you
may wish to consider not using a scheme which
requires that password/passphrase. In this
case, using a no-passphrase public key for
SSH/SFTP. Storing a datum like this in a
file tends to defeat any added security which
it might otherwise add to the scheme.

I normally use a no-passphrase public key for
SSH (and seldom use SFTP), so I know nothing,
but I would expect any interactive passkey
request to be made explicitly to the user's
terminal, not to any batch file, if only to
discourage users from storing passdata in a
file.

> where [lf} is the ascii line feed command
> (in editor I used CTRL-V then CTRL-J)

If I thought that the LF was important, I'd
create/convert to a Record format: Stream_LF
file, instead of trying to add delimiters to
a file which already has some kind of record
separator scheme.

[Thomas Ritter]

SFTP from VMS to non-VMS usually trips up in the file format of the keys generated. There maybe a conversion step required on the other end to make sure the keys are in the correct format. VMS is really foreign to these UNIX hosts !
What is the other end ? We maybe able to help.
Are you able to SSH to the remote end ?

Whose implementation of SFTP are you using ?

[John Gillings]

Erik,

Is there a reason you can't use a public key? Once that's configured properly, both SSH and SFTP work between systems without the need to feed passwords in.

As Thomas said, the biggest hurdle is getting the key file in the right format across different platforms. I don't think there's any universal advice. I think I tried generating keys at both ends, compared key files, and CONVERTed the file from the VMS end to look like the one at other end.

That said, have you tried changing your characters into s in the batch file? I'd also try it without quotes.

I've also found that SSH and SFTP seem to care about where SYS$COMMAND comes from. In some cases immediately preceeding the command with:

$ DEFINE/USER SYS$COMMAND NL:

has solved odd behaviour.

[Steven Schweda]

> Is there a reason you can't use a public
> key? Once that's configured properly, both
> SSH and SFTP work between systems without
> the need to feed passwords in.

That depends on how you created the key,
which can be done with or without a
passphrase. For example, without:

alp $ ssh alp
Authentication successful.
@ SYS$MANAGER:ANNOUNCE.TXT [Great software.]
[...]

Or with:

alp $ ssh -i identification._pp alp
Passphrase for key "/ALP$DKA0/sms/ssh2/SMS_ID_DSA_1024_A" with comment "1024-bit
dsa, sms@alp.antinode.org, Fri Jun 27 2003 03:57:52":

Authentication successful.
@ SYS$MANAGER:ANNOUNCE.TXT
[...]


> As Thomas said, the biggest hurdle [...]

I'll admit that it would be nice to know that
a plain SSH works properly before diving into
messy SFTP procedures.

[Mårten Holmberg]

Hi,
I don't know about Erik's environment, but in our case the keyfile is given to us from the host, so we can't have one without passphrase.

The example I gave works for us, but if I could choose, I'd use a keyfile without a passphrase.

Regarding the [lf] and/or stream_lf, you just have to test until you find the "thing" that works between your client and that host. And be prepared to test again if they, or you, applies any patches etc. to the sftp client/server.

/MÃ¥rten

[Erik Harclerode]

I have created a new key without a password and sent it over to the person on the receiving end. I just got word that they have it all loaded so I will attempt to connect again in batch mode and see what happens.

My only concern now is that their system requires a user name and password to login so I'm wondering if this will work from the batch file or not. I guess I'll find out shortly!

Thanks for your continued support!

[Erik Harclerode]

Ok, good news.... once I created a new key without a password I am able to authenticate without a problem. Now I am unable to get it to read my batch file for the account password. Any ideas on how to pass this password in?

[Steven Schweda]

> Now I am unable to get it to read my batch
> file for the account password. Any ideas on
> how to pass this password in?

Which "account password"? What, exactly, are
you doing, and with what? Around here:

alp $ sftp sms@alp
sftp> pwd
ALP$DKA0:[SMS]
sftp> quit


Once I'm in, I'm in. I have no idea where
I'd put another password.

Batch file, shmatch file. Can you make this
task go interactively? When you can, provide
a transcript, and then we can worry about how
to automate it further.

[Erik Harclerode]

Hi,
I've always been able to run it interactively but now I'm trying to automate the login process so I can then automate an upload each day.

The password I'm referencing is one given to me from the receiver. They gave me a user name and password to access their system.

here is what it looks like:

$ sftp plmweb
This is a private system.
Unauthorized access is prohibited and will
userid@www.plmweb.org's password:
sftp> ls
.
Download
sftp>

[Steven Schweda]

> [...] once I created a new key without a
> password I am able to authenticate without
> a problem.

Apparently not. Perhaps you could provide a
transcript showing how you arrived at that.
Or does "without a problem" not imply
"without a password"?

> $ sftp plmweb
> This is a private system.
> Unauthorized access is prohibited and will
> userid@www.plmweb.org's password:

That would seem not to be using a key which
works without a password. (Or any key at
all. It looks like "password"
authentication, not "hostbased" or
"publickey".)

Adding "-v" to your SSH or SFTP command
should show you more about which
authentication method were attempted, and
which one actually worked, along with other
potentially interesting details.

[Erik Harclerode]

Hi Steve,
Initially I had a password on my SSH key, so when I SFTP'd to the server it would ask me for two sets of information. First it would ask me for a password for my SSH key. Once authenticated there it would ask me for a different account password, which is completely UNRELATED to the SSH key. This account password was provided by the people I'm trying to connect to.

Now, I've created a new SSH key WITHOUT a password and am able to connect to their system and not have to enter my SSH password, but it still asks for the account password. So in a sense there are two levels of security, one with the SSH key and another with a simple user name and password login. The new SSH key without the password gets me through the first door but now I need to figure out a way to feed the account password to the prompt so I can get completely onto their system.

I hope this clears things up.
Thanks,
Erik

[Richard Whalen]

Since it is still asking you for the account password my first guess is that the public key authentication is failing. Try doing an SSH with /DEBUG=4, if you see a line containing "Method 'publickey' disabled.", then you'll have confirmation that it is just moving from one authentication method to the next.

[Erik Harclerode]

I have confirmed with the people that I'm connecting with that the account password is required after authentication.

[Richard Whalen]

If account password is required, then they must have a line equivalent to this in their configuration file:

RequiredAuthentications password

Unfortunately, you are not going to get it to work in batch mode in this case.

[Erik Harclerode]

Ok, closing this thread. I finally got it to work. I convinced the people to let me have a single authentication and remove the account password piece of this problem. Now I am able to login without any problems.

thanks to all who helped!

[Steven Schweda]

> I convinced [...]

If you told me that the extra password
would end up being stored in a file
somewhere, then I'd sure be willing to forgo
the extra "security".

[Erik Harclerode]

Solution Found!