ssh upgrade --login prompt takes time

[Rajendra prasad NVR]

I upgraded ssh version
before install:
Secure_Shell A.05.10.006 HP-UX Secure Shell
After install
Secure_Shell A.05.20.004 HP-UX Secure Shell

Os version is B.11.11 and model9000/800/rp7420

After upgrade it was taking 1 to 2 minutes to get login prompt. Please advice if we need to change any configuration in ssh after install.

[Johnson Punniyalingam · Accepted solution]

would look into why ssh-rand-helper is hanging around for such a long time. It generates random number by running commands from /opt/ssh/etc/ssh_prng_cmds.

Any of the commands from ssh_prng_cmds can be a cause for slowdown. For example if /var/adm/wtmp is large, /usr/bin/last will take a while to run.

Check what files ssh-rand-helper has opened with lsof or attach to it with tusc to see what it's up to.

Thanks,
Johnson

[Ganesan R]

Hi,

Are you sure that login take long time exactly after ssh upgrade?

How about other logins like telnet or rlogin?

is the server running without any resource bottleneck?

You could also try ssh -vv username@server to get the debug messages.

[Rajendra prasad NVR]

The server performance is good.

With lsof i am not able to find any process
(ttetimqa:/opt/ssh/etc)# lsof |grep -i ssh_prng_cmds
(ttetimqa:/opt/ssh/etc)#
=============================================

(ttetimqa:/opt/ssh/etc)# more ssh_prng_cmds
# entropy gathering commands

# Format is: "program-name args" path rate

# The "rate" represents the number of bits of usuable entropy per
# byte of command output. Be conservative.
#
# $Id: ssh_prng_cmds,v 1.1.1.1 2007/02/06 05:50:41 cvsuser Exp $

"ls -alni /usr/adm" /usr/bin/ls 0.02
"ls -alni /etc/mail" /usr/bin/ls 0.02
"ls -alni /usr/mail" /usr/bin/ls 0.02
"ls -alti /usr/adm/syslog" /usr/bin/ls 0.02
"ls -alti /var/adm/syslog" /usr/bin/ls 0.02
"ls -alti /usr/bin" /usr/bin/ls 0.02
"ls -alti /usr/tmp" /usr/bin/ls 0.02
"ls -alti /opt" /usr/bin/ls 0.02
"ps -al" /usr/bin/ps 0.03
"ps -ex" /usr/bin/ps 0.03
"who am i" /usr/bin/who 0.01
"vmstat" /usr/bin/vmstat 0.01
"tail -100 /var/adm/syslog" /usr/bin/tail 0.01
"tail -100 /var/adm" /usr/bin/tail 0.01
"tail -100 /var/adm/syslog/mail.log" /usr/bin/tail 0.01
"ls -alni /dev/log" /usr/bin/ls 0.02
"ls -alni /var/adm" /usr/bin/ls 0.02
"ls -alni /var/adm/syslog" /usr/bin/ls 0.02
"ls -alni /usr/adm/syslog" /usr/bin/ls 0.02
"ls -alni /usr/bin" /usr/bin/ls 0.02
"ls -alni /tmp" /usr/bin/ls 0.02
"ls -alni /var/tmp" /usr/bin/ls 0.02
"ls -alni /usr/tmp" /usr/bin/ls 0.02
"ls -alti /dev/log" /usr/bin/ls 0.02
"ls -alti /var/adm" /usr/bin/ls 0.02
"ls -alti /etc/mail" /usr/bin/ls 0.02
"ls -alti /tmp" /usr/bin/ls 0.02
"ls -alti /var/tmp" /usr/bin/ls 0.02
"netstat -an" /usr/bin/netstat 0.05
"ps laxww" /usr/bin/ps 0.03
"ps -efl" /usr/bin/ps 0.03
"w" /usr/bin/w 0.05
"who -u" /usr/bin/who 0.01
"last" /usr/bin/last 0.01
"last log" /usr/bin/last 0.01
"uptime" /usr/bin/uptime 0.01
"ipcs -a" /usr/bin/ipcs 0.01
"tail -100 /var/adm/syslog/syslog.log" /usr/bin/tail 0.01
"tail -100 /var/adm/syslog/syslog.log" /usr/bin/tail 0.01
#"sar -d 1 2" /usr/sbin/sar 0.04
#"netstat -rn" /usr/bin/netstat 0.05
#"netstat -n" /usr/bin/netstat 0.05
#"netstat -s" /usr/bin/netstat 0.05
#"netstat -is" /usr/bin/netstat 0.05
#"arp -a" /usr/sbin/arp 0.02

[Rajendra prasad NVR]

telnet and rlogin are blocked in this servers.

[Fredrik.eriksson]

Actually, I've run into this a couple of times before and my problems have always related to reverse lookups of connecting address.

There is a config option in your sshd.conf that looks like this:

UseDNS no

This is per default commented out because this feature is default "yes".

Best regards
Fredrik Eriksson

[Jitesh purohit_1]

What application are you using to SSH in? if you use putty and it has a log as well. Right click the heading and click event log. That can give you an idea of what's happening

Jitesh

[Steven E. Protter]

Shalom,

First thing I always do to eliminate common cause is this:

try the ssh login via hostname, then numeric ip.

If its faster via numeric ip, it signals a DNS problem. If you are not using DNS another common problem is an HP-UX server needs an /etc/hosts entry for itself on the server side so it does self lookup in a reasonable period of time.

You see ssh runs a number of integrity checks including makeing sure a hostname login matches the system.

If I log into a system ssh -vvv server2, part of the login process on the server is to make sure the server sees itself as server2.

It might still accept the login but if nslookup server2 does not provide a good answer, it really slows down the login process.

The other possibility is bad secure shell software. The only way to deal with that is to report the problem to HP via the http://software.hp.com website and wait for a new version.

As it so happens, I just spent my Sunday resurrecting my two D class systems in preparation for building a Service Guard Cluster. One of the things I did not get to is updating Secure Shell. Currently the website is not working for me, but as soon as it is, I will install it and report test results back to you.

SEP

[Rajendra prasad NVR]

Thank you for all giving the suggestions.

Fredrik,
In sshd.conf the line is UseDNS yes is commented.
I replace with UseDNS no. But still resolved.

Jitesh,
I am using putty software. Looks normal in event logs.

Steven,
We are using only ip address in putty to login with ssh.

It is taking now 30 sec to login hope the up gradation version is like that.
Earlier version it used to take 5 to 10 sec.

[Mel Burslan]

I am not sure what changed between the two versions of ssh you mention, but go to another unix server and ssh to this server using the -vvv option, i.e.,

ssh -vvv my_ssh_upgraded_server

and post the debug output here for further help. What it sounds like, it is either looking for an extra authentication which was not there before, a timeout value somehow got changed.

Also helpful is posting your /opt/ssh/etc/sshd_conf file contents here.

[Bill Hassell]

30 seconds (and multiples 60 and 90) typicallt signal name resolution issues. On your server run nsquery and nslookup for the incoming or local system where ssh is being run:

nsquery hosts local-IP-address
nslookup local-IP-address

If the IP address cannot be resolved (technically, authenticated), you'll see a 30 second delay for each DNS server listed in /etc/resolv.conf. Check /etc/nsswitch.conf for these lines:

hosts: files [NOTFOUND=continue UNAVAIL=continue] dns
ipnodes: files [NOTFOUND=return] dns

If dns is first, try putting files first, then add the IP address (and dummy name) of your local machine to /etc/hosts and try ssh again.

[Rajendra prasad NVR]

Bill,

I checked all your suggetions but not resolved the problem.

Mel,

Below are lines uncommented in sshd_config file

[server8:/opt/ssh/etc]grep -v '#' sshd_config
Protocol 2
PermitRootLogin no
KerberosAuthentication yes
UsePAM yes
X11Forwarding yes
X11UseLocalhost no
EnforceSecureTTY yes
Subsystem sftp /opt/ssh/libexec/sftp-server


When ssh -vvv is working fine means with no time it is givinig promt. I am using putty 0.60 which was downloaded from
http://www.chiark.greenend.org.uk/~sgtatham/putty/

[Rajendra prasad NVR]

ssh -vvv i treid to login from one server to other server it is working fine. but problem when i try to login from my desk top to server directly.

But other 11.23 version unix servers i am able to login with no issue. The problem where i am facing is 11.11 version hpunix servers.

[Bill Hassell]

> login from my desk top to server directly.

Since server to server is OK, these boxes probably have a validated IP address from your DNS server. Since only your desktop (Linux or a PC?) fails, it is likely that your server does not know your IP address.

[Rajendra prasad NVR]

Looks like it is not issue with server.

[Wiktor Cieślak]

Hi,

I had same problem on some servers, in my case it was related to wtmps file. On one of the servers wtmps file was 1,8GB big and login took over 1 min. After I have zeroing the wtmps file, login takes ~2 sec.
You can use script below to archive and clear wtmps file:

/usr/sbin/acct/fwtmp < /var/adm/wtmps > $SOME_DIR/wtmp_`date +%Y%m%d`.txt

cat /dev/null > /var/adm/wtmps

Pozdrawiam

[balaji_vvv]

usr/sbin/acct/fwtmp < /var/adm/wtmps > $SOME_DIR/wtmp_`date +%Y%m%d`.txt

cat /dev/null > /var/adm/wtmps

wtmps? or wtmpx?

[balaji_vvv]

usr/sbin/acct/fwtmp < /var/adm/wtmps > $SOME_DIR/wtmp_`date +%Y%m%d`.txt

cat /dev/null > /var/adm/wtmps

wtmps or wtmpx?