ssh & service guard

[Merv Lumarque]

We are currently running a number of processes which use ssh without being prompted for a password.

The problem occurs when in failover mode. Instead of our processes sending files to box X with an IP address of 10.10.10.1 it now attempts to send files to box X with an IP address of 10.10.10.2 and the following message is displayed:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is 5e:e0:25:a4:50:c3:cf:e6:6e:da:8f:cc:74:0f:bf:2d.
Please contact your system administrator.
Add correct host key in /home/user1/.ssh/known_hosts to get rid of this message.
Offending key in /home/user1/.ssh/known_hosts:2 RSA host key for X has changed and you have requested strict checking.
Host key verification failed.
lost connection

Any ideas on how to solve this problem?

Thanks in advance.

[Alexander Chuzhoy]

login to the station as the user you're tryng to reach it via ssh.
go to $HOME/.ssh directory where $HOME is the home dir of this user,i.e. if the user is root then it would be /root/.ssh
Edit the file known_hosts find the entry for previous IP (10.10.10.1 in you case)
and delete it.

[Merv Lumarque]

In non-failover in need to connect to X server on IP 10.10.10.1 but in fail-over mode in need to connect to X server on IP 10.10.10.2.
I know I can remove the entry in my known_hosts file but this only fixes it temporarily. I will need to remove the entry in the known_hosts file again when I fail the box over. Is there a way to connect to a different box in failover without having to modify the known_hosts file?

[Steven E. Protter]

You need to clear the entries in the known_hosts files and possibly exchange public keys again.

As far as SG goes, if all ssh traffic to the SG cluster is to a floating IP address, these errors should not occur any more after taking the steps above.

SEP

[Emil Velez · Accepted solution]

I have had a customer make both nodes from his cluster use the same keys for ssh. So what you do is install SSH on both nodes. Copy the keys from 1 node to all of the other nodes. This way the same encryption key is used for all systems. Then when your client systems ssh into the systems they see the same key.

good luck

[Merv Lumarque]

Thanks for everyone's suggestions. What we decided to do is modify the StrictHostKeyChecking parameter in the /opt/ssh/etc/ssh_config file directory during failover and set it to no. Yes, this is a security issue and we have communicated this to the business.

Thanks Again.

[Serviceguard for Linux]

Can you clarify?

It is not clear what processes are the ones that failed over. From the description, it is also not clear why the IP address is changing.

So are the processes that are doing the ssh the ones that failover? So can you please describe what is running on the cluster nodes and what is on the client?

Also, after failover, has the ssh "restarted" or is there an assumption that the ssh should still be connected? If there is an assumption that ssh is still connected, that could be the problem.

[Serviceguard for Linux]

By the way, there is a secific "serviceguard" category under Linux covering Serviceguard for Linux. http://forums1.itrc.hp.com/service/forums/categoryhome.do?categoryId=555

[Armin Kunaschik]

As Emil said before:
The ONLY way of not getting into this problem
is to share one hostkey (/etc/ssh/ssh_host*)
for both servers.
Simply copy all 3 host keys to the other system.
Thus, the floating IP has always the same host key... everything is fine :-)

This will be no security issue because the maping is OK, you can activate strict checking again.

Armin

[Bob_Vance]

If you are doing the ssh to the *floating* IP, this problem can't solved by simply copying keys and the known_hosts entry.

Changing StrictHostKeyChecking doesn't help either, nor does changing CheckHostIP.
At least it didn't for me.

When the package moves to the other node, then the key fingerprint changes (for the floating name/IP) and the ssh to the floating IP fails with this error.

It will still work fine for access to the *node* names/IPs, but *not* the floating name/IP.

I wanted to be able to determine dynamically the hostname from where the package is currently running by doing

# pkgHost=$( ssh pkg-name/IP hostname )

to be able to do special things (like backup things) dynamically.
I ended up allowing 'remsh' *only* between the nodes. Then

# pkgHost=$( remsh pkg-name/IP hostname )

works fine, of course.


bv