ssh does not work when ssh client and server are on the same subnet

[newa]

Hi,

I just got several hp rp3440 servers moved to my location. Those servers came with fully installed hp-ux 11i v2 and had been in use for years.

After I put them on the network and assigned them the new ip/submask etc... I found out I could use Putty to ssh to those servers from subnets other than the one those servers are on. BUT SSH FAILED TO CONNECT TO THE SERVER WHEN THE SSH CLIENT WAS ON THE SAME SUBNET AS THOSE SERVERS!!!???

I checked the ipFilter setting and sshd_config and could not find the answer. DID I MISS ANYTHING? WHAT ELSE SHOULD I CHECK?

THANKS IN ADVANCE!

************************************

To check ipFilter I did this:

h466:/roothome # ipf -V
ipf: HP IP Filter: v3.5alpha5 (A.11.23.15.01) (376)
Kernel: HP IP Filter: v3.5alpha5 (A.11.23.15.01)
Running: yes
Log Flags: 0 = none set
Default: pass all, Logging: available
Active list: 1
h466:/roothome #
**********************************************
h466:/roothome # ipfstat -ioh
empty list for ipfilter(out)
empty list for ipfilter(in)
h466:/roothome #

**********************************************
Here is the sshd_config: (see the attachement for better format :) )

# $OpenBSD: sshd_config,v 1.70 2004/12/23 23:11:00 djm Exp $

# This is the sshd server system-wide configuration file. See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/bin:/bin:/usr/sbin:/sbin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented. Uncommented options change a
# default value.

#Port 22
Protocol 2
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

# HostKey for protocol version 1
#HostKey /opt/ssh/etc/ssh_host_key
# HostKeys for protocol version 2
#HostKey /opt/ssh/etc/ssh_host_rsa_key
#HostKey /opt/ssh/etc/ssh_host_dsa_key

# Lifetime and size of ephemeral version 1 server key
#KeyRegenerationInterval 1h
#ServerKeyBits 768

# Logging
#obsoletes QuietMode and FascistLogging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
PermitRootLogin yes
#StrictModes yes
#MaxAuthTries 6
#CountKeyAuthBadLogins no

# Auth selection
#
#HostbasedAuthAllowUsers
#HostbasedAuthDenyUsers
#PubkeyAuthAllowUsers
#PubkeyAuthDenyUsers
#KerberosAuthAllowUsers
#KerberosAuthDenyUsers
#KerberosOrLocalPasswdAllowUsers
#KerberosOrLocalPasswdDenyUsers
PasswordAuthallowUsers
#PasswordAuthDenyUsers
#ChallRespAuthAllowUsers [pam] user1 user2 ...
#ChallRespAuthDenyUsers [pam] user1 user2 ...
#ChallRespAuthAllowUsers [bsdauth] user1 user2 ...
#ChallRespAuthDenyUsers [bsdauth] user1 user2 ...
#ChallRespAuthAllowUsers [skey] user1 user2 ...
#ChallRespAuthDenyUsers [skey] user1 user2 ...
#ChallRespAuthAllowUsers [securid] user1 user2 ...
#ChallRespAuthDenyUsers [securid] user1 user2 ...
#GSSAPIAuthAllowUsers
#GSSAPIAuthDenyUsers

#RSAAuthentication yes
#PubkeyAuthentication yes
#AuthorizedKeysFile .ssh/authorized_keys

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#RhostsRSAAuthentication no
# similar for protocol version 2
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# RhostsRSAAuthentication and HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication yes
#PermitEmptyPasswords no

# Change to no to disable s/key passwords
#ChallengeResponseAuthentication yes

# Kerberos options
KerberosAuthentication yes
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication mechanism.
# Depending on your PAM configuration, this may bypass the setting of
# PasswordAuthentication, PermitEmptyPasswords, and
# "PermitRootLogin without-password". If you just want the PAM account and
# session checks to run without PAM authentication, then enable this but set
# ChallengeResponseAuthentication=no
UsePAM no

#AllowTcpForwarding yes
#GatewayPorts no
X11Forwarding yes
#X11DisplayOffset 10
X11UseLocalhost no
#PrintMotd yes
#PrintLastLog yes
#TCPKeepAlive yes
#EnforceSecureTTY no
#UseLogin no
#UsePrivilegeSeparation yes
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS yes
#PidFile /var/run/sshd.pid
#MaxStartups 10

# no default banner path
Banner /etc/issue

# override default of no subsystems
Subsystem sftp /opt/ssh/libexec/sftp-server

# sftp-server logging
#LogSftp no
#SftpLogFacility AUTH
#SftpLogLevel INFO

# sftp-server umask control
#SftpUmask

#SftpPermitChmod yes
#SftpPermitChown yes

****************************************

THE END

[Michael Steele_2]

Hi

Try to login as a normal user and see what happens. By default the root cannot ssh into a machine unless you uncomment "PermitRootLogin yes"

Verify the FQDN of your server and nslookup both ip and FQDN and hostname. Adjust your /etc/hosts files and verify DNS records.

Check syslog for errors and post.

[Victor Fridyev]

In addition to above, please check whether setting in /etc/rc.config.d/netconf and /etc/hosts coinside.

HTH

[Steven Schweda]

> [...] SSH FAILED TO CONNECT [...]

As usual, it might help if you showed actual
commands with their actual output instead of
vague descriptions or interpretations.

Adding "-v" to an "ssh" command can also be
useful.

> [...] and assigned them the new ip/submask
> etc... [...]

> [...] DID I MISS ANYTHING? [...]

The answer to that question might depend on
what's in that "etc".

[newa]

Thank you all. As I said in my original posting: THE SSH WORKED WELL WHEN THE CLIENT AND THE SERVER ARE ON DIFFERENT SUBNET. IT JUST STOPPED WORKING WHEN THE BOTH ARE ON THE SAME SUBNET.

When I tried it I used ip address.
Today I am home and can not give you the failed ssh example because via VPN I am on a different subnet from the hp server and ssh works well. I will post the failed ssh command tomorrow when I get back to the server room.

When I put the server on the network I used "set_parms initial" to set up the ip, subnet mask, default gateway ip and I did not set up DNS name and NDS server ip

Again thanks a lot.


Here are the configs

***** /etc/rc.config.d/netconf ****
SUDO>h466-$ cat /etc/rc.config.d/netconf
HOSTNAME="h466"
OPERATING_SYSTEM=HP-UX
LOOPBACK_ADDRESS=127.0.0.1

INTERFACE_NAME[0]=lan0
IP_ADDRESS[0]=172.20.123.230
SUBNET_MASK[0]=255.255.0.0
BROADCAST_ADDRESS[0]=""
INTERFACE_STATE[0]=up
DHCP_ENABLE[0]=0

ROUTE_DESTINATION[0]=default
ROUTE_MASK[0]=""
ROUTE_GATEWAY[0]=172.20.0.1
ROUTE_COUNT[0]=1
ROUTE_ARGS[0]=""

GATED=0
GATED_ARGS=""


RDPD=0

RARPD=0

IP_ADDRESS[2]=2.171.120.162
SUBNET_MASK[2]=255.255.0.0
INTERFACE_NAME[2]=lan2

ROUTE_DESTINATION[2]="net 2.0.0.0"
ROUTE_MASK[2]="255.0.0.0"
ROUTE_GATEWAY[2]="2.171.1.1"
ROUTE_COUNT[2]="1"
BROADCAST_ADDRESS[2]=2.171.255.255
INTERFACE_STATE[2]=up
SUDO>h466-$
********************* end **************

********* /etc/hosts ***************
SUDO>h466-$ cat /etc/hosts
## Configured using SAM by root on Tue Oct 10 11:03:58 2006
## Configured using SAM by root on Wed Dec 10 13:20:24 2008
172.20.123.230 h466
127.0.0.1 localhost loopback
SUDO>h466-$
***************** end *******************
************* /etc/resolv.conf *******
SUDO>uguth466-$ cat /etc/resolv.conf
#domain vent.com
#nameserver 172.20.13.50
SUDO>uguth466-$
****************** end **********************

[Steven Schweda]

> [...] IT JUST STOPPED WORKING [...]

YES, AND IF WE KNEW EXACTLY WHAT "STOPPED
WORKING" MEANT, THEN WE MIGHT BE ABLE TO
SUGGEST WHY.

> [...] can not give you the failed ssh
> example [...]

Can you SSH from the HP-UX system to itself?
(It is on its own subnet, right?)

> [...] and I did not set up DNS name and NDS
> server ip

Huh? Does that mean that name and address
look-ups work or not?

[newa]

The ssh failed to connect to 127.0.0.1

h460:/etc/default # ssh -v dliu@127.0.0.1
OpenSSH_4.5p1+sftpfilecontrol-v1.1-hpn12v14, OpenSSL 0.9.7l 28 Sep 2006
HP-UX Secure Shell-A.04.50.021, HP-UX Secure Shell version
debug1: Reading configuration data /opt/ssh/etc/ssh_config
debug1: Connecting to 127.0.0.1 [127.0.0.1] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/3
debug1: identity file /roothome/.ssh/id_rsa type -1
debug1: identity file /roothome/.ssh/id_dsa type -1
ssh_exchange_identification: Connection closed by remote host
uguth460:/etc/default # ifconfig lan0
lan0: flags=1843
inet 172.20.123.224 netmask ffff0000 broadcast 172.20.255.255
h460:/etc/default # ssh -v dliu@172.20.123.224
OpenSSH_4.5p1+sftpfilecontrol-v1.1-hpn12v14, OpenSSL 0.9.7l 28 Sep 2006
HP-UX Secure Shell-A.04.50.021, HP-UX Secure Shell version
debug1: Reading configuration data /opt/ssh/etc/ssh_config
debug1: Connecting to 172.20.123.224 [172.20.123.224] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/3
debug1: identity file /roothome/.ssh/id_rsa type -1
debug1: identity file /roothome/.ssh/id_dsa type -1
ssh_exchange_identification: Connection closed by remote host
h460:/etc/default #

[Steven Schweda]

> ssh_exchange_identification: Connection closed by remote host

Have you looked in the system log file(s)?

Have you tried generating new host keys after
changing all the network stuff?


> OpenSSH_4.5p1+sftpfilecontrol-v1.1-hpn12v14, OpenSSL 0.9.7l 28 Sep 2006

Probably not the latest version, but old age
is probably not the biggest problem here.

[Michael Steele_2]

debug1: identity file /roothome/.ssh/id_rsa type -1
debug1: identity file /roothome/.ssh/id_dsa type -1


Your connection is established, it clearlys states connection established, but then it fails on your rsa key, which is your public key, and your private key, or id_dsa key. Here, the authentication fails.

Looks like you need to regen the keys including the new hostname's network changes.

See testing section of this link.

http://gaarai.com/2009/02/19/ssh-tutorial-for-ubuntu-linux/

[Steven Schweda]

> debug1: identity file /roothome/.ssh/id_rsa type -1
> debug1: identity file /roothome/.ssh/id_dsa type -1
>
> Your connection is established, it clearlys
> states connection established, but then it
> fails on your rsa key, which is your public
> key, and your private key, or id_dsa key.
> Here, the authentication fails.

Not really. The SSH client did not find two
potential expected key files, but that's long
before it attempted to do any user
authentication. These messages are entirely
harmless, and are unrelated to the actual problem.

Consider, for example, the following
transcript of a successful session
establishment:

dy # uname -a
HP-UX dy B.11.11 U 9000/785 2012616114 unlimited-user license

dy # ssh -V
OpenSSH_5.3p1+sftpfilecontrol-v1.3-hpn13v5, OpenSSL 0.9.8l 5 Nov 2009
HP-UX Secure Shell-A.05.30.007, HP-UX Secure Shell version

dy # ssh -v -l sms alp-l
OpenSSH_5.3p1+sftpfilecontrol-v1.3-hpn13v5, OpenSSL 0.9.8l 5 Nov 2009
HP-UX Secure Shell-A.05.30.007, HP-UX Secure Shell version
debug1: Reading configuration data /opt/ssh/etc/ssh_config
debug1: Connecting to alp-l [10.0.0.9] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/3
debug1: identity file /root/.ssh/identity type -1
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_dsa type -1

[This newer SSH version looks for three key
files, none of which exists here. Then it
continues, beginning the conversation with
the remote system. Note that in a working
configuration, these messages are harmless,
not fatal. What's not working in the problem
case is the stuff below, the handshaking
between the two systems.]

debug1: Remote protocol version 2.0, remote software version 3.2.0 SSH OpenVMS V
5.5 VMS_sftp_version 3
debug1: no match: 3.2.0 SSH OpenVMS V5.5 VMS_sftp_version 3
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3p1+sftpfilecontrol-v1.3-hpn13v5
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: AUTH STATE IS 0
debug1: REQUESTED ENC.NAME is 'aes128-cbc'
debug1: kex: server->client aes128-cbc hmac-md5 none
debug1: REQUESTED ENC.NAME is 'aes128-cbc'
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: sending SSH2_MSG_KEXDH_INIT
debug1: expecting SSH2_MSG_KEXDH_REPLY
debug1: Host 'alp-l' is known and matches the DSA host key.
debug1: Found key in /root/.ssh/known_hosts:10
debug1: ssh_dss_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received

@ SYS$MANAGER:ANNOUNCE.TXT

[That's the (defective) greeting from the
remote system. Basic handshaking is now
complete. _Now_ user authentication begins.]

debug1: Authentications that can continue: hostbased,publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/identity
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Trying private key: /root/.ssh/id_dsa

[Again, the expected key files are not found,
so publickey authentication fails, so we move
along to the next authentication method,
password...]

debug1: Next authentication method: password
sms@alp-l's password:
debug1: Authentication succeeded (password).

[I provided the correct password, so password
authentication succeeded, and so the
interactive session begins...]

debug1: Final hpn_buffer_size = 131072
debug1: HPN Disabled: 1, HPN Buffer Size: 131072
debug1: channel 0: new [client-session]
debug1: Entering interactive session.

Welcome to VMS (Alpha) V8.3 on ALP.
[...]



> ssh_exchange_identification: Connection closed by remote host

While this does not show a user
authentication problem, it does suggest some
configuration problem, possibly involving the
old host keys, as previously suggested.

[Steven Schweda]

> Have you looked in the system log file(s)?

On my 11.11 system, I see SSH-related
messages in "/var/adm/syslog/syslog.log".
(Mine show success, but I'd still expect to
find useful info there if there's a problem.)

[newa]

Thanks all for your help. I found out the culprit was tcp wrapper (/etc/hosts.allow file)

On the server I did:

sshd -d -d -d -r -p 2222

on the same box I did:

ssh -vv -p 2222 127.0.01

The the server console wrote to console:
connection refused by tcp wrapper.

I got the idea from openSSH O'reily book. :)

[newa]

Closing it

[Steven Schweda]

> On the server I did:
>
> sshd -d -d -d -r -p 2222
> [...]
> The the server console wrote to console:
> [...]

And before, I'd guess that it was writing a
similar complaint to the log file, but if you
don't look, then you're unlikely to find.