topic Re: SFTP in batch mode - Re: thread 985928 in Operating System - OpenVMS

SFTP in batch mode - Re: thread 985928

Erik Harclerode — Tue, 12 Feb 2008 16:21:49 GMT

Hi,
My apologies in advance for the long thread but I tried to be thorough and answer as many questions up front. I've done lots of searching on these forums and it seems that the definitive thread for SFTP in batch mode is this thread:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=985928

After looking it over I still have a few questions.

First, my situation is that I'm trying to FTP from OpenVMS to an external Unix server. I, the client, am set up to accept all forms of authentication. The server is set up to accept publickey and password authentication. I am able to connect successfully manually, but not through batch mode. When I do connect manually it asks for 2 passwords: first the key password and then the account password.

I've created a file named batchtest.txt located in a drive named DSA106:[programmer]. I've also set up my IDENTIFICATION. file to point to the proper key as well as the SSH2_CONFIG. file to be set up with the host I'm trying to connect to, I'll call it "testhost" in this example.

I try to connect as follows at the $ prompt:
SFTP "-B" DSA106:[programmer]batchtest.txt testhost

batchtest.txt looks as follows:
"""[lf]

where [lf} is the ascii line feed command (in editor I used CTRL-V then CTRL-J)

After executing my SFTP command I receive this error:
Authentication failed.
Disconnected; authentication cancelled by user (Authentication cancelled by user
.).
FATAL: ssh2 client failed to authenticate. (or you have too old ssh2 installed,
check with ssh2 "-V")

%TCPIP-F-SSH_FATAL, non-specific fatal error condition

Here is further information from -v:
debug: server offers auth methods 'publickey,password'.
debug: Ssh2AuthPubKeyClient/AUTHC-PUBKEY.C:1682: adding keyfile "/DSA106/program
mer/ssh2/ID_DSA_2048_A" to candidates
debug: Constructing and sending signature in publickey authentication.
debug: Ssh2AuthPubKeyClient/AUTHC-PUBKEY.C:874: ssh_client_auth_pubkey_send_sign
ature: reading /DSA106/programmer/ssh2/ID_DSA_2048_A
debug: Ssh2Common/SSHCOMMON.C:185: DISCONNECT received: Authentication cancelled
by user.

I've tried everything I could find online and have had no success, so I was hoping someone else could shed some light on this situation for me. What am I doing wrong?

I also notice in the 985928 thread that he uses something called login.com. I have a login.com on the system but there is nothing in it, so I don't see how it would add value to my batch mode.

Any help would be appreciated.

Re: SFTP in batch mode - Re: thread 985928

Volker Halle — Tue, 12 Feb 2008 17:12:26 GMT

Erik,

I have been able to set up hostbased authentication based on the SSH manual pointed to in the other thread more than 2 years ago. I've never used it again since, so I would have to start all over now...

Maybe the fundamental misunderstanding is, that you can't use passwords stored in files with SFTP in batch mode. You need to configure hostbased authentication and this needs to be done both on the SSH client and SSH server.

Volker.

Re: SFTP in batch mode - Re: thread 985928

Mårten Holmberg — Tue, 19 Feb 2008 12:18:21 GMT

Hi,
we uses SFTP in batch with publickey and not hostbased authentication.

This is how we got it to work.

$ ! Load key-pair to memory
$ ! -----------------------------------
$ SET NOVER ! Hide passwords from log
$ @sys$manager:tcpip$define_commands
$ open/write add_file temp.file
$ write add_file "$ssh_add -p id_dsa_ssh2"
$ write add_file "''pwd'"
$ close add_file
$
$ ssh_agent
$ wait 00:00:10 ! Wait for SSH_AGENT to start
$ @temp.file
$ delete/log temp.file;*
$ REST_VERIFY ! Restore Verify state
$ ssh_add -l ! List the key we loaded
$
... create sftp-batchfile...
... convert it to streamLF...
$
$ sftp "-B" sftp-batch.file "''usr'"@'host'
$

We get have a password database that we call to get the symbols HOST, USR and PWD and the symbol REST_VERIFY is setup to restore the verify state to what it was before we used the password.

Try a test batch-file with something like:
ls[lf]
exit[lf]
in it at first.

I'm not sure that the wait for the SSH_AGENT is needed, but when we exerimented to get this to work we sometimes got an agent error.

Hope this helps

/MÃ¥rten

Re: SFTP in batch mode - Re: thread 985928

Steven Schweda — Tue, 19 Feb 2008 16:12:45 GMT

In any situation where you would need to
store a password/passphrase in a file, you
may wish to consider not using a scheme which
requires that password/passphrase. In this
case, using a no-passphrase public key for
SSH/SFTP. Storing a datum like this in a
file tends to defeat any added security which
it might otherwise add to the scheme.

I normally use a no-passphrase public key for
SSH (and seldom use SFTP), so I know nothing,
but I would expect any interactive passkey
request to be made explicitly to the user's
terminal, not to any batch file, if only to
discourage users from storing passdata in a
file.

> where [lf} is the ascii line feed command
> (in editor I used CTRL-V then CTRL-J)

If I thought that the LF was important, I'd
create/convert to a Record format: Stream_LF
file, instead of trying to add delimiters to
a file which already has some kind of record
separator scheme.

Re: SFTP in batch mode - Re: thread 985928

Thomas Ritter — Tue, 19 Feb 2008 22:55:32 GMT

SFTP from VMS to non-VMS usually trips up in the file format of the keys generated. There maybe a conversion step required on the other end to make sure the keys are in the correct format. VMS is really foreign to these UNIX hosts !
What is the other end ? We maybe able to help.
Are you able to SSH to the remote end ?

Whose implementation of SFTP are you using ?

Re: SFTP in batch mode - Re: thread 985928

John Gillings — Tue, 19 Feb 2008 23:46:54 GMT

Erik,

Is there a reason you can't use a public key? Once that's configured properly, both SSH and SFTP work between systems without the need to feed passwords in.

As Thomas said, the biggest hurdle is getting the key file in the right format across different platforms. I don't think there's any universal advice. I think I tried generating keys at both ends, compared key files, and CONVERTed the file from the VMS end to look like the one at other end.

That said, have you tried changing your characters into s in the batch file? I'd also try it without quotes.

I've also found that SSH and SFTP seem to care about where SYS$COMMAND comes from. In some cases immediately preceeding the command with:

$ DEFINE/USER SYS$COMMAND NL:

has solved odd behaviour.

Re: SFTP in batch mode - Re: thread 985928

Steven Schweda — Wed, 20 Feb 2008 05:32:32 GMT

> Is there a reason you can't use a public
> key? Once that's configured properly, both
> SSH and SFTP work between systems without
> the need to feed passwords in.

That depends on how you created the key,
which can be done with or without a
passphrase. For example, without:

alp $ ssh alp
Authentication successful.
@ SYS$MANAGER:ANNOUNCE.TXT [Great software.]
[...]

Or with:

alp $ ssh -i identification._pp alp
Passphrase for key "/ALP$DKA0/sms/ssh2/SMS_ID_DSA_1024_A" with comment "1024-bit
dsa, sms@alp.antinode.org, Fri Jun 27 2003 03:57:52":

Authentication successful.
@ SYS$MANAGER:ANNOUNCE.TXT
[...]

> As Thomas said, the biggest hurdle [...]

I'll admit that it would be nice to know that
a plain SSH works properly before diving into
messy SFTP procedures.

Re: SFTP in batch mode - Re: thread 985928

Mårten Holmberg — Wed, 20 Feb 2008 07:46:24 GMT

Hi,
I don't know about Erik's environment, but in our case the keyfile is given to us from the host, so we can't have one without passphrase.

The example I gave works for us, but if I could choose, I'd use a keyfile without a passphrase.

Regarding the [lf] and/or stream_lf, you just have to test until you find the "thing" that works between your client and that host. And be prepared to test again if they, or you, applies any patches etc. to the sftp client/server.

/MÃ¥rten

Re: SFTP in batch mode - Re: thread 985928

Erik Harclerode — Wed, 20 Feb 2008 13:19:32 GMT

I have created a new key without a password and sent it over to the person on the receiving end. I just got word that they have it all loaded so I will attempt to connect again in batch mode and see what happens.

My only concern now is that their system requires a user name and password to login so I'm wondering if this will work from the batch file or not. I guess I'll find out shortly!

Thanks for your continued support!

Re: SFTP in batch mode - Re: thread 985928

Erik Harclerode — Wed, 20 Feb 2008 18:51:44 GMT

Ok, good news.... once I created a new key without a password I am able to authenticate without a problem. Now I am unable to get it to read my batch file for the account password. Any ideas on how to pass this password in?

Re: SFTP in batch mode - Re: thread 985928

Steven Schweda — Wed, 20 Feb 2008 18:59:43 GMT

> Now I am unable to get it to read my batch
> file for the account password. Any ideas on
> how to pass this password in?

Which "account password"? What, exactly, are
you doing, and with what? Around here:

alp $ sftp sms@alp
sftp> pwd
ALP$DKA0:[SMS]
sftp> quit

Once I'm in, I'm in. I have no idea where
I'd put another password.

Batch file, shmatch file. Can you make this
task go interactively? When you can, provide
a transcript, and then we can worry about how
to automate it further.

Re: SFTP in batch mode - Re: thread 985928

Erik Harclerode — Wed, 20 Feb 2008 20:11:23 GMT

Hi,
I've always been able to run it interactively but now I'm trying to automate the login process so I can then automate an upload each day.

The password I'm referencing is one given to me from the receiver. They gave me a user name and password to access their system.

here is what it looks like:

$ sftp plmweb
This is a private system.
Unauthorized access is prohibited and will
userid@www.plmweb.org's password:
sftp> ls
.
Download
sftp>

Re: SFTP in batch mode - Re: thread 985928

Steven Schweda — Wed, 20 Feb 2008 22:54:38 GMT

> [...] once I created a new key without a
> password I am able to authenticate without
> a problem.

Apparently not. Perhaps you could provide a
transcript showing how you arrived at that.
Or does "without a problem" not imply
"without a password"?

> $ sftp plmweb
> This is a private system.
> Unauthorized access is prohibited and will
> userid@www.plmweb.org's password:

That would seem not to be using a key which
works without a password. (Or any key at
all. It looks like "password"
authentication, not "hostbased" or
"publickey".)

Adding "-v" to your SSH or SFTP command
should show you more about which
authentication method were attempted, and
which one actually worked, along with other
potentially interesting details.

Re: SFTP in batch mode - Re: thread 985928

Erik Harclerode — Thu, 21 Feb 2008 13:26:38 GMT

Hi Steve,
Initially I had a password on my SSH key, so when I SFTP'd to the server it would ask me for two sets of information. First it would ask me for a password for my SSH key. Once authenticated there it would ask me for a different account password, which is completely UNRELATED to the SSH key. This account password was provided by the people I'm trying to connect to.

Now, I've created a new SSH key WITHOUT a password and am able to connect to their system and not have to enter my SSH password, but it still asks for the account password. So in a sense there are two levels of security, one with the SSH key and another with a simple user name and password login. The new SSH key without the password gets me through the first door but now I need to figure out a way to feed the account password to the prompt so I can get completely onto their system.

I hope this clears things up.
Thanks,
Erik

Re: SFTP in batch mode - Re: thread 985928

Richard Whalen — Thu, 21 Feb 2008 15:58:38 GMT

Since it is still asking you for the account password my first guess is that the public key authentication is failing. Try doing an SSH with /DEBUG=4, if you see a line containing "Method 'publickey' disabled.", then you'll have confirmation that it is just moving from one authentication method to the next.

Re: SFTP in batch mode - Re: thread 985928

Erik Harclerode — Thu, 21 Feb 2008 16:14:06 GMT

I have confirmed with the people that I'm connecting with that the account password is required after authentication.

Re: SFTP in batch mode - Re: thread 985928

Richard Whalen — Thu, 21 Feb 2008 16:29:31 GMT

If account password is required, then they must have a line equivalent to this in their configuration file:

RequiredAuthentications password

Unfortunately, you are not going to get it to work in batch mode in this case.

Re: SFTP in batch mode - Re: thread 985928

Erik Harclerode — Thu, 21 Feb 2008 17:56:31 GMT

Ok, closing this thread. I finally got it to work. I convinced the people to let me have a single authentication and remove the account password piece of this problem. Now I am able to login without any problems.

thanks to all who helped!

Re: SFTP in batch mode - Re: thread 985928

Steven Schweda — Thu, 21 Feb 2008 18:28:16 GMT

> I convinced [...]

If you told me that the extra password
would end up being stored in a file
somewhere, then I'd sure be willing to forgo
the extra "security".

Re: SFTP in batch mode - Re: thread 985928

Erik Harclerode — Thu, 21 Feb 2008 18:28:28 GMT

Solution Found!