FTP from UNIX client (C-Kermit) to Gene6 w/SSL

[Andre Stewart]

Firstly, I'm new to the whole SSL/TLS process. I'm used to doing all of my encrypted communications using SSH.

I need to FTP from UNIX (HP-UX) to a Gene6 server while using SSL authentication/encryption.

I've been given what I think is a self-signed, x509 certificate, in file that is in DEF format, from the administrator of the Gene6 server.

On the UNIX client, I have OpenSSL v0.9.7e and C-Kermit 8.0.211 (built with SSL support).

I've converted the DER file to PEM format, and I now have 2 files. For illustration purposes, let's call them:

/tmp/cert.der
/tmp/cert.pem

How can I verify that the certificate is indeed self-signed (the contents can be view with OpenSSL commands)?

How can I use either versions of this certificate to connect and encrypt my FTP session to the Gene6 server?

Can it be setup to auto-authenticate (not use a password) with the certificate?

I've tried what I think is the appropriate process but receive the following as output:


With the DER file:

(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth ssl rsa-cert-file /tmp/cert.der
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth tls debug on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth tls verbose on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth ssl debug on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth ssl verbose on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set ftp verbose on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set ftp authtype ssl tls
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set ftp debug on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>show auth
Authentication: Kerberos 4 (not installed)
Authentication: Kerberos 5 (not installed)
Authentication: SSL/TLS (OpenSSL 0.9.7e 25 Oct 2004)
RSA Certs file: /tmp/cert.der
RSA Certs Chain file: (none)
RSA Key file: /tmp/cert.der
DSA Certs file: (none)
DSA Certs Chain file: (none)
DH Key file: (none)
DH Param file: (none)
CRL file: (none)
CRL dir: (none)
Random file: (none)
Verify file: (none)
Verify dir: (none)
Cipher list: HIGH:MEDIUM:LOW:+ADH:+EXP
DHE-RSA-AES256-SHA
DHE-DSS-AES256-SHA
AES256-SHA
EDH-RSA-DES-CBC3-SHA
EDH-DSS-DES-CBC3-SHA
DES-CBC3-SHA
DHE-RSA-AES128-SHA
DHE-DSS-AES128-SHA
AES128-SHA
DHE-DSS-RC4-SHA
RC4-SHA
RC4-MD5
EXP1024-DHE-DSS-DES-CBC-SHA
EXP1024-DES-CBC-SHA
EXP1024-RC2-CBC-MD5
EDH-RSA-DES-CBC-SHA
EDH-DSS-DES-CBC-SHA
DES-CBC-SHA
EXP1024-DHE-DSS-RC4-SHA
EXP1024-RC4-SHA
EXP1024-RC4-MD5
EXP-EDH-RSA-DES-CBC-SHA
EXP-EDH-DSS-DES-CBC-SHA
EXP-DES-CBC-SHA
EXP-RC2-CBC-MD5
EXP-RC4-MD5
Certs OK? no
Debug mode: off
Verbose mode: off
Verify mode: peer-cert
SSL only? no
TLS only? no
Authentication: SRP (not installed)
Authentication: NTLM (not installed)
(/var/tmp/C-Kermit8.0.211/) C-Kermit>ftp open /ssl xxxx.xxxx.xxx 21 /user:ZZZ
SSL DEBUG ACTIVE
Loading RSA certificate into SSL
Error loading certificate from /tmp/cert.der
=>START SSL/TLS connect on COMMAND
SSL_handshake:UNKWN before/connect initialization
SSL_connect:UNKWN before/connect initialization
SSL_connect:3WCH_A SSLv3 write client hello A
SSL_write_alert
SSL_connect:error in 3RSH_A SSLv3 read server hello A
ftp: SSL/TLS connect COMMAND error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
=>DONE SSL/TLS connect on COMMAND
?Can't FTP connect to xxxx.xxxx.xxx:21
(/var/tmp/C-Kermit8.0.211/) C-Kermit>exit

With the PEM file:

UNIX_hostname:/var/tmp/C-Kermit8.0.211# ./kermit
Executing /usr/share/lib/kermit/ckermit.ini for UNIX...
Good Afternoon!
C-Kermit 8.0.211, 10 Apr 2004, for HP-UX 11.00
Copyright © 1985, 2004,
Trustees of Columbia University in the City of New York.
Type ? or HELP for help.
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth ssl rsa-cert-file /tmp/cert.pem
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth tls debug on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth tls verbose on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth ssl debug on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth ssl verbose on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set ftp verbose on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set ftp authtype ssl tls
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set ftp debug on
(/var/tmp/C-Kermit8.0.211/) C-Kermit>>show auth
Authentication: Kerberos 4 (not installed)
Authentication: Kerberos 5 (not installed)
Authentication: SSL/TLS (OpenSSL 0.9.7e 25 Oct 2004)
RSA Certs file: /tmp/cert.pem
RSA Certs Chain file: (none)
RSA Key file: /tmp/cert.pem
DSA Certs file: (none)
DSA Certs Chain file: (none)
DH Key file: (none)
DH Param file: (none)
CRL file: (none)
CRL dir: (none)
Random file: (none)
Verify file: (none)
Verify dir: (none)
Cipher list: HIGH:MEDIUM:LOW:+ADH:+EXP
DHE-RSA-AES256-SHA
DHE-DSS-AES256-SHA
AES256-SHA
EDH-RSA-DES-CBC3-SHA
EDH-DSS-DES-CBC3-SHA
DES-CBC3-SHA
DHE-RSA-AES128-SHA
DHE-DSS-AES128-SHA
AES128-SHA
DHE-DSS-RC4-SHA
RC4-SHA
RC4-MD5
EXP1024-DHE-DSS-DES-CBC-SHA
EXP1024-DES-CBC-SHA
EXP1024-RC2-CBC-MD5
EDH-RSA-DES-CBC-SHA
EDH-DSS-DES-CBC-SHA
DES-CBC-SHA
EXP1024-DHE-DSS-RC4-SHA
EXP1024-RC4-SHA
EXP1024-RC4-MD5
EXP-EDH-RSA-DES-CBC-SHA
EXP-EDH-DSS-DES-CBC-SHA
EXP-DES-CBC-SHA
EXP-RC2-CBC-MD5
EXP-RC4-MD5
Certs OK? no
Debug mode: on
Verbose mode: on
Verify mode: peer-cert
SSL only? no
TLS only? no
Authentication: SRP (not installed)
Authentication: NTLM (not installed)
(/var/tmp/C-Kermit8.0.211/) C-Kermit>ftp open /ssl xxxx.xxxx.xxx 21 /user:ZZZ
SSL DEBUG ACTIVE
Loading RSA certificate into SSL
Error loading key from /tmp/cert.pem
=>START SSL/TLS connect on COMMAND
SSL_handshake:UNKWN before/connect initialization
SSL_connect:UNKWN before/connect initialization
SSL_connect:3WCH_A SSLv3 write client hello A
SSL_write_alert
SSL_connect:error in 3RSH_A SSLv3 read server hello A
ftp: SSL/TLS connect COMMAND error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
=>DONE SSL/TLS connect on COMMAND
?Can't FTP connect to xxxx.xxxx.xxx:21
(/var/tmp/C-Kermit8.0.211/) C-Kermit>

I noticed in both cases that the Certificate was not consdered ok.
Am I not properly referencing it with Kermit? Does it need to be installed a specificate root certificate store?

Any direction/advice would be greatly appreciated.

[Steven E. Protter]

Shalom,

You are trying to use ftp which is inherently insecure due to unencrypted authentication in a secure way.

Yes, it can be done but its a lot of work.

See if this device can use openssh secure shell. If so the secure shell client for HP-UX is free and works very well. Both the data stream and authentication are encrypted.

SEP

[Wyman Wong Man Wai]

If the file is x509 cert, try
$ openssl x509 -in cert.pem -text
If the file is rsa key, try
$ openssl rsa -in cert.pem -text

[Heironimus]

I've never used C-Kermit for ftps, but I can offer a little information on SSL. In general I would agree with the recommendation to ask about ssh, different FTP over SSL implementations don't always work together. If they really like SSL then HTTP is a better option because it's more properly standardized.

SSL is essentially a public key encryption system. You have a private key (usually an RSA key) and a certificate, which is actually the public key. It's not uncommon to put the key and certificate in a single file, I assume that's what you have. For analysis it might be easier to copy the key and cert out to different files, if you have a PEM file it's just a copy/paste.

The private key is normally protected with DES or 3DES encryption, which is why you have to enter a password. You can, however, use an unencrypted key so that you won't need a password for automated transfers. openssl should be able to extract the cleartext key like this: "openssl rsa -in ssl.key -out unencrypted.key". Make sure to secure the unencrypted key properly.

You can find out if the certificate is self-signed with openssl: "openssl x509 -noout -text -in cert.pem" If the Issuer and Subject are the same then it's self-signed.

[Andre Stewart]

Yeah, I've already floated the SSH ballon, but the Windoze admins. would have no parts of it, from either a "setup a server" perspective, or come and get the data (me being the SSH server) perspective.

I think from a Windoze, point-of-view, they may have a problem with scheduling a perpetual job to get the data from me. There may be some security issues involved (Does the process run as a service? If so, who is the user? Does it need to be a local account?, etc?) Whatever all the issues are, I'm constrained to the model I'm presenting.

The results from the OpenSSL queries of the certificate files are thus:

The x509 query was successful. The output looks good (omitted for security purposes), but the RSA query yielded the following (I also did a DSA):

UNIX_hostname:/# openssl rsa -in /tmp/cert.pem -text
unable to load Private Key
19816:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:637:Expecting: ANY PRIVATE KEY
UNIX_hostname:/# openssl dsa -in /tmp/cert.pem -text
read DSA key
unable to load Key
19823:error:0906D06C:PEM routines:PEM_read_bio:no start line:pem_lib.c:637:Expecting: ANY PRIVATE KEY


I find that odd since the x509 query output contained:

Certificate:
Data:
Version: 3 (0x2)
Serial Number: 0 (0x0)
Signature Algorithm: md5WithRSAEncryption


and:

Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (1024 bit)
Modulus (1024 bit):


and:


Exponent: 3 (0x3)
Signature Algorithm: md5WithRSAEncryption


So, did the RSA/DSA queries fail because I could not contact a proper root certificate store to validate the certificate file? I didn't think that was necessary for a self-signed certificate.


Andre

[Andre Stewart]

I've just run across something that seems interesting. It's found at http://www.columbia.edu/kermit/security81.html (search on "x509" - third occurance):


If you wish to provide support for authentication of clients using public key certificates you must provide two custom functions X509_to_user() and X509_userok. These functions provide the certificate to local userid mapping and user authorization functionality. Example functions that use the /UID field of the Certificate Subject name may be activated by specifying:


make entry KFLAGS=-DX509_UID_TO_USER

when compiling C-Kermit. If you with to use the Certificate Subject Alternate Name you can specify:


make entry KFLAGS=-DX509_SUBJECT_ALT_NAME_TO_USER

The X509_to_user() and X509_userok() functions are the last functions in the ck_ssl.c module. See Appendix III.


I didn't build it using these flags. It they refer to Linux, but I'm running HP-UX 11i v1 on a PA-RISC (rp4440) machine.

Ideas?

Andre

[Heironimus]

It looks like the rsa/dsa queries failed because you don't have a private key in that file. In a PEM file (it's just plain text, you can cat or more it) a certificate starts with the line "-----BEGIN CERTIFICATE-----" and ends with the line "-----END CERTIFICATE-----". An RSA private key starts with the line "-----BEGIN RSA PRIVATE KEY-----" and ends with the line "-----END RSA PRIVATE KEY-----". If you don't have both then you can't do both queries on that file and you can't use that file to establish a connection.

It's probably worthwhile to repeat those two tests on the original file they sent you. It's possible that both pieces just didn't make it in to the PEM version, if the original file contained both key and cert you would probably have had to convert each one seperately.

SSL isn't simple, but it's not really all that complex either. It's just that you don't have somebody to sit down with you and work through it. If you have any experienced web server admins they might be able to help you at least figure what you've got and what you need.

[Wyman Wong Man Wai]

It seems the PEM lacks the key.
PEM is text header wrapped DER.
You may open both PEM and DER to check if you lost rsa/dsa key during DER-PEM conversion.

SSL connection needs CA cert, client cert and client key on the client side. You may check the output of 'openssl x509 ...' to see which CA is used to sign the cert in DER. If it is self-signed, you need to obtain the CA cert form the Gene6 sysadmin. If it is signed by public CA, you may use ftp://kermit.columbia.edu/kermit/c-kermit/ca_certs.pem.

Config the Kermit by
SET AUTH SSL VERIFY-FILE path/ca_certs.pem
set auth tls rsa-cert-file path/cert.pem
set auth tls rsa-key-file path/key.pem

[Andre Stewart]

Thanks for the heads up about the file containing both a key and a certificate.

I do have a section at the bottom with

-----BEGIN CERTIFICATE-----
.
.
.
-----END CERTIFICATE-----


I cut that part out and put that in a file called /tmp/cert_only.pem. I put the remaining part in a file called /tmp/key_only.pem

I then issued the following in C-Kermit:

(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth ssl rsa-cert-file /tmp/cert_only.pem
(/var/tmp/C-Kermit8.0.211/) C-Kermit>set auth ssl rsa-key-file /tmp/key_only.pem

but a "show auth" still says my certificat is NOT ok. An attempt to FTP yeilds:

(/var/tmp/C-Kermit8.0.211/) C-Kermit>ftp open /ssl xxxx.xxxx.xxx 21 /user:ZZZ
SSL DEBUG ACTIVE
Loading RSA certificate into SSL
Error loading key from /tmp/key_only.pem
=>START SSL/TLS connect on COMMAND
SSL_handshake:UNKWN before/connect initialization
SSL_connect:UNKWN before/connect initialization
SSL_connect:3WCH_A SSLv3 write client hello A
SSL_write_alert
SSL_connect:error in 3RSH_A SSLv3 read server hello A
ftp: SSL/TLS connect COMMAND error: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
=>DONE SSL/TLS connect on COMMAND
?Can't FTP connect to XXX.XXX.XXX.XXX:21
(/var/tmp/C-Kermit8.0.211/) C-Kermit>

So, I'm still stuck.

Andre

[Wyman Wong Man Wai]

The error begins at "Error loading key from /tmp/key_only.pem". If you cannot get "openssl rsa -in key_only.pem -text" right, the file does not work. Do you find "-----BEGIN RSA PRIVATE KEY-----" as stated by Heironimus.

Since the problem is still not solved yet, let's start at the begining.
1. Ask the administrator of the Gene6 server what the file he provide is. It may be just their simple careless mistake.
2. Do not obtain the cert and key files as plain text through email. I experienced that some webmail corrupt the base64 encoding in the cert and key files. And I once emailed three files, CA cert, client cert and client key to another, but the CA cert did not show up in the attachment (in win32 email client)! So if you really received the cert/key through email, ask them to ZIP it before transmission.
3. Ask the sender to provide md5sum of the original files and verify it after you receive it.
4. What is the method you used to convert DER to PEM?
5. I remind you that you need three things, CA cert, client cert and client key.

[Andre Stewart]

No, I don't have the -- BEGIN RSA CERTIFICAT - part. Maybe the .der file isn't what I really need.

I'm still not clear on how to evaluate the .der file to dtermine what it really is and if I need to re-request it with specific qalities.

The one thing I do know is that my combined .pem file seems to be x509, at least, openssl doesn't give me an error with that query. It actually gives me the contents of the .pem file.

The contents are similar to this:

Certificate:
Data:
Version: 3 (0x2)
Serial Number: 0 (0x0)
Signature Algorithm: md5WithRSAEncryption
Issuer: L=REMOTE LOCATION, ST=XX, C=US, CN=HOST.FOOBAR.NET/emailAddress=ISSUER@FOOBAR.NET, O=BIG A CORPORATION, OU=Montana Operations
Validity
Not Before: Jan 24 15:02:54 2007 GMT
Not After : Jan 24 15:02:54 2017 GMT
Subject: L=REMOTE LOCATION, ST=XX, C=US, CN=HOST.FOOBAR.NET/emailAddress=patriciaISSUER@FOOBAR.NET, O=BIG A CORPORATION, OU=Montana Operations
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (1024 bit)
Modulus (1024 bit):
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00
Exponent: 3 (0x3)
Signature Algorithm: md5WithRSAEncryption
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:
00:00

-----BEGIN CERTIFICATE-----
.
.
.
-----END CERTIFICATE-----

[Michael Stringer · Accepted solution]

I with with Andre, and here's how we fixed the issue...

C-Kermit will act as a SSL/TLS client when compiled with OpenSSL installed. The caveat to compiling is making sure you have curses installed - Kermit has a special file transfer status window that needs those libraries.

Once compiled, here are the commands to make a SSL/TLS connection to an FTP server:

# ./kermit
C-Kermit 8.0.211, 10 Apr 2004, for HP-UX 11.00
Copyright (C) 1985, 2004,
Trustees of Columbia University in the City of New York.
Type ? or HELP for help.
(/root/) C-Kermit> set ftp authtype tls
(/root/) C-Kermit> ftp

At this point, Kermit will attempt the SSL connection, and if the SSL certificate provided by the server is self signed, out-of-date, or has some other issue, you will get the details of the cert and a prompt to continue. There's a command to turn that off:

(/root/) C-Kermit> set authentication tls verify no

..this allows Kermit to just "trust" the certificate handed to it - requiring a high level of trust in the connection to the SFTP site.

From there, Kermit will give you details on the SSL cert's encryption, and show that the connection is secured.

Then it's a login prompt.

while not as secure as SSH, it's very usable when you do not have control of the ftp server and Security is breathing down your neck to not send passwords in the clear.

[Andre Stewart]

Yes, Michael is correct. As a matter of fact, he should be fully credited with the solution.

To give more detail:

I somehow got misdirected early on with the idea that I needed to have a certificate local to the client in order to connect to the Gene6 server. I was beating my head against the wall trying to get my C-Kermit/OpenSSL implementation to recognize the certificate in the form of an actual file. Although, that may ultimately be possible, it was unnecessary.

Michael properly understood that, upon connecting to the Gene6 server, the server would send the C-Kermit FTP client a public certificate which, upon acceptance by the client, would be used to encrypt the FTP session from that point forward.

Now that he's figured that out, I've been able to used the robust scripting features of Kermit to finish my project.

[Andre Stewart]

See post just pevious to this one.