topic Re: IPSec between HP - SUN servers in Operating System - HP-UX

IPSec between HP - SUN servers

Fredric Vådegård — Wed, 09 Feb 2005 07:30:06 GMT

Hi,
have any one successfully configured IPSec for a HP-UX 11.00 and a Solaris 9 server? Will use this for backup thru a firewall. The test do not include the FW, that is the next step.

The problem seems to be on the Sun box since I have successfully configured and connected to a W2K server from the HP box. There are no GUI for the IPSec on Solaris. I have followed an example from the "IPsec and IKE Administration Guide" from Sun (http://docs-pdf.sun.com/817-2694/817-2694.pdf).
When starting up the in.iked process in debug mode, I see that there are problem in Phase 1 (Phase 1 negotiation error: No proposal chosen)

Thanks!

//Fredric

Re: IPSec between HP - SUN servers

Peter Godron — Wed, 09 Feb 2005 08:26:01 GMT

Hi,
could you double-check your IKE policies.
That message would appear if policies have a security or encryption mismatch.
Are there any other messages before or after?
Regards

Re: IPSec between HP - SUN servers

Fredric Vådegård — Wed, 09 Feb 2005 09:00:04 GMT

Hi,

now I havn't the same problem, but it still donÂ´t work.

I can see "ESP SPI=0x323 Replay=1" when snooping (Replay is counting up by one every) and running a telnet against the remote IP-address.

Currently I use the following files
/etc/inet/ipsecinit.conf
/etc/inet/ike/config
/etc/inet/ipseckey
removed the /etc/inet/secret/ike.preshared, because it was not in the new example of setup I saw on http://builder.com.com/5100-6372-1044095.html

//Fredri

Re: IPSec between HP - SUN servers

Slawomir Gora — Wed, 09 Feb 2005 09:14:54 GMT

Hi,

I'am using IPSEC between Solaris 9 and HPUX 11.0 and 11.11 but only in transport mode.
Maby you have problem with your preshared password - on Solaris it must be hex.

Can you attach yours configs ?
HPUX: /var/adm/ipsec/polices.txt
Sol9: /etc/inet/ike/config
/etc/inet/ipsecinit.conf

Re: IPSec between HP - SUN servers

Fredric Vådegård — Wed, 09 Feb 2005 09:47:03 GMT

I have attached the requested files.

HP:
server name - hpbup
Below I have listed what is used for the setup of HP/SUN in the policies.txt
begin filter hpbup_setest02
begin oakley isakmp_hpbup2
begin transform ESP-3DES-HMAC-SHA1

Thanks!

//Fredric

Re: IPSec between HP - SUN servers

Slawomir Gora — Wed, 09 Feb 2005 13:40:04 GMT

Hi,

I think that your problem is in oakley group for in.iked daemon.
On HPUX you have oakley group 2
On Solaris you have oakley group 5
Your solaris polices are in old style try use my config:

#########################
/etc/inet/ipsecinit.conf

{ saddr 192.36.176.142
daddr 192.36.176.216
} ipsec {
encr_auth_algs md5
encr_algs 3des
sa shared
}

#########################
/etc/inet/iked/config
p1_lifetime_secs 28800
p2_lifetime_secs 28800

{
label "hpbup_setest02"
local_id_type ip
local_addr 192.36.176.142
remote_addr 192.36.176.216

p1_xform {
auth_method preshared
oakley_group 2
auth_alg md5
# encr_alg 3des-cbc - YOU CAN TRY THIS
encr_alg 3des
p1_lifetime_secs 28800
}
p2_lifetime_secs 28800
}

#####################
If you want convert key from text to hex use my script:

#!/bin/sh

if [ "$1" = "" ]
then
echo "usage `basename $0` keyfile"
exit
fi

OUTFILE="$1.hex"

if [ -f $1 ]
then
cat $1 | od -X | \
awk '{printf "%s%s%s%s",$2,$3,$4,$5} END {printf "\n"}' |\
sed -e 's/0a000000//g' > ${OUTFILE}

echo "Key is stored in ${OUTFILE}"
else
echo "No such file $1"
fi

#end of file

Re: IPSec between HP - SUN servers

Slawomir Gora — Wed, 09 Feb 2005 13:46:51 GMT

Hi,

if you will have problems try to debug iked daemon:

ikeadm -n set debug all /tmp/iked.log

I know you can have problem with your communication when you will restart or flush keys on solaris site - no information is send to HP and HP will send encrypted packages to solaris and no negotiation until KEY TIMEOUT (in your example 28800) will happen.

Re: IPSec between HP - SUN servers

Andrew Cowan — Thu, 10 Feb 2005 02:31:01 GMT

IPSec is also very fussy about the declarion of system hostnames, and IP-Addresses. Ensure that both machines really know each other by the exact canonical or alias hostnames, and that if you have amulti-homed hosts, data is arriving through the correct interfaces.

Re: IPSec between HP - SUN servers

Fredric Vådegård — Thu, 10 Feb 2005 04:24:43 GMT

Thank you for your help so far!

I have changed the configuration files as you suggested.
I donÂ´t get any error msg when starting up the in.iked process, it even says "added rule hpbup_setest02", "config_update succeeded!".
But, as you wrote, I donÂ´t get any answear from hpbup when trying to telnet hpbup from setest02(SUN).
Should I wait 8h (28800sec) and see if works after the time_out?

//Fre

Re: IPSec between HP - SUN servers

Andrew Cowan — Thu, 10 Feb 2005 04:58:41 GMT

This may sound really stupid but there isn't any IP netmask issues here are there? Re my earlier point: If one of the machines has the wrong netmask, or can be reached on a couple of interfaces, then IP-SEC will fail during the negotiation stage. The other thing you can try is to go to the lowest level of link first e.g. start with AH and no ESP, and see if that works with a really basic secret/key e.g. "test". If successful keep upping the authentication levels until it fails.

Another idea. There aren't any hidden control characters or trailing spaces or tabs in your config files are there?

Re: IPSec between HP - SUN servers

Fredric Vådegård — Thu, 10 Feb 2005 05:16:36 GMT

Hi,

I have rebooted the SUN box (setest02).
I have attached the iked.log file, hopefully it is useful.

I will check/test a more basic connection.

//Fredric

Re: IPSec between HP - SUN servers

Slawomir Gora — Thu, 10 Feb 2005 05:51:11 GMT

Hi,

I have one question do you need to configure
all trafic between HP and SUN ?
I always configure only selected services (telnet, ssh etc..).

Re: IPSec between HP - SUN servers

Fredric Vådegård — Thu, 10 Feb 2005 06:17:01 GMT

I will use the connection for backup (Data Protector, port 5555).

Re: IPSec between HP - SUN servers

Andrew Cowan — Thu, 10 Feb 2005 06:23:02 GMT

From this error message it looks like IP-SEC is unsure of what kind of IKE key to expect. Are you sure that you are not missing a parameter in the config file that explicity mentions a static key, and its format e.g. ASCII or HEX.

I have seen this problem when connecting between two AIX boxes when using the GUI and entering exactly the same data on both. The only solution was to do all the configuration on ONE node and then export it. The export reverses the data. You could see if either Solaris or HP-UX provides such a utility and examine the XML files for strange characters or extra parameters?

Re: IPSec between HP - SUN servers

Fredric Vådegård — Thu, 10 Feb 2005 06:52:03 GMT

Hi,

I had renamed the /etc/inet/secret/ike.preshared file. The file is now renamed. I assume that I must have the ike.preshared file, it holds the key. But still it don't work!

I would like to double check with you to see if I doing right after I have changed some of the config files.

1) ipsecconf -f
2) ipsecconf -a /etc/inet/ipsecinit.conf
3) pkill in.iked
4) /usr/lib/inet/in.iked -d
5) ikeadm -n set debug all /tmp/iked.log

I had a config file earlier called /etc/inet/ipseckey, that removed. Should I have that file?

I appreciate your help!

//Fredric

Re: IPSec between HP - SUN servers

Fredric Vådegård — Thu, 10 Feb 2005 11:52:34 GMT

YES - IT WORKS !!!!!!!!!!!

I tried between 2 SUN boxes and get it working at once. Then I found out how to audit the trafic on the HP box and I got more information that lead me on the right target.

Thanks!

//Fredric