HPE Community read-only access December 15, 2018
This is a maintenance upgrade. You will be able to read articles and posts, but not post or reply.
Hours:
Dec 15, 4:00 am to 10:00 am UTC
Dec 14, 10:00 pm CST to Dec 15, 4:00 am CST
Dec 14, 8:00 pm PST to Dec 15, 2:00 am PST
cancel
Showing results for 
Search instead for 
Did you mean: 

SSH Very Loaded

 
SOLVED
Go to solution
AZayed
Super Advisor

SSH Very Loaded

Dears,
Good day,

Recently we force our users to switch to SFTP instead of FTP. Now, every time I login to the system using SSH it takes 10 seconds after entering the password. I think this is related to the last action. I tried to reduce the logging of sftp-server to QUIET but this is didn't help.

So, can you help me tune this issue?

Thanks
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
9 REPLIES
Mel Burslan
Honored Contributor

Re: SSH Very Loaded

try posting the full output (screen capture) from command :

ssh -vvv my_hostname_here

and mark the line in this output where it hangs for that 10 seconds on this output
________________________________
UNIX because I majored in cryptology...
AZayed
Super Advisor

Re: SSH Very Loaded

Hi Mel,
Hope you doing well.

It stops at the end of the ssh login. After the last line.

wtmpx@wtmpx-laptop:~$ ssh -vvv -l root XXX.XXX.XXX.YYY
OpenSSH_5.5p1 Debian-4ubuntu4, OpenSSL 0.9.8o 01 Jun 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to XXX.XXX.XXX.YYY [XXX.XXX.XXX.YYY] port 22.
debug1: Connection established.
debug1: identity file /home/wtmpx/.ssh/id_rsa type -1
debug1: identity file /home/wtmpx/.ssh/id_rsa-cert type -1
debug1: identity file /home/wtmpx/.ssh/id_dsa type -1
debug1: identity file /home/wtmpx/.ssh/id_dsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_5.6p1+sftpfilecontrol-v1.3-hpn13v7
debug1: match: OpenSSH_5.6p1+sftpfilecontrol-v1.3-hpn13v7 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.5p1 Debian-4ubuntu4
debug2: fd 3 setting O_NONBLOCK
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa-cert-v00@openssh.com,ssh-dss-cert-v00@openssh.com,ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit: none,zlib@openssh.com,zlib
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: kex_parse_kexinit: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: kex_parse_kexinit: ssh-rsa,ssh-dss
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc@lysator.liu.se
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: hmac-md5,hmac-sha1,umac-64@openssh.com,hmac-ripemd160,hmac-ripemd160@openssh.com,hmac-sha1-96,hmac-md5-96
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit: none,zlib@openssh.com
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit:
debug2: kex_parse_kexinit: first_kex_follows 0
debug2: kex_parse_kexinit: reserved 0
debug2: mac_setup: found hmac-md5
debug1: kex: server->client aes128-ctr hmac-md5 none
debug2: mac_setup: found hmac-md5
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug2: dh_gen_key: priv key bits set: 139/256
debug2: bits set: 519/1024
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: check_host_in_hostfile: host XXX.XXX.XXX.YYY filename /home/wtmpx/.ssh/known_hosts
debug3: check_host_in_hostfile: host XXX.XXX.XXX.YYY filename /home/wtmpx/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 5
debug1: Host 'XXX.XXX.XXX.YYY' is known and matches the RSA host key.
debug1: Found key in /home/wtmpx/.ssh/known_hosts:5
debug2: bits set: 508/1024
debug1: ssh_rsa_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /home/wtmpx/.ssh/id_rsa ((nil))
debug2: key: /home/wtmpx/.ssh/id_dsa ((nil))
debug1: Authentications that can continue: publickey,password,keyboard-interactive,hostbased
debug3: start over, passed a different list publickey,password,keyboard-interactive,hostbased
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /home/wtmpx/.ssh/id_rsa
debug3: no such identity: /home/wtmpx/.ssh/id_rsa
debug1: Trying private key: /home/wtmpx/.ssh/id_dsa
debug3: no such identity: /home/wtmpx/.ssh/id_dsa
debug2: we did not send a packet, disable method
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 1
Password:
debug3: packet_send2: adding 32 (len 23 padlen 9 extra_pad 64)
debug2: input_userauth_info_req
debug2: input_userauth_info_req: num_prompts 0
debug3: packet_send2: adding 48 (len 10 padlen 6 extra_pad 64)
debug1: Authentication succeeded (keyboard-interactive).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug2: callback start
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug1: Sending environment.
debug3: Ignored env ORBIT_SOCKETDIR
debug3: Ignored env SSH_AGENT_PID
debug3: Ignored env TERM
debug3: Ignored env SHELL
debug3: Ignored env XDG_SESSION_COOKIE
debug3: Ignored env WINDOWID
debug3: Ignored env GNOME_KEYRING_CONTROL
debug3: Ignored env GTK_MODULES
debug3: Ignored env USER
debug3: Ignored env LS_COLORS
debug3: Ignored env SSH_AUTH_SOCK
debug3: Ignored env DEFAULTS_PATH
debug3: Ignored env SESSION_MANAGER
debug3: Ignored env USERNAME
debug3: Ignored env XDG_CONFIG_DIRS
debug3: Ignored env DESKTOP_SESSION
debug3: Ignored env PATH
debug3: Ignored env PWD
debug3: Ignored env GDM_KEYBOARD_LAYOUT
debug1: Sending env LANG = en_US.utf8
debug2: channel 0: request env confirm 0
debug3: Ignored env GNOME_KEYRING_PID
debug3: Ignored env MANDATORY_PATH
debug3: Ignored env GDM_LANG
debug3: Ignored env GDMSESSION
debug3: Ignored env SPEECHD_PORT
debug3: Ignored env SHLVL
debug3: Ignored env HOME
debug3: Ignored env GNOME_DESKTOP_SESSION_ID
debug3: Ignored env LOGNAME
debug3: Ignored env XDG_DATA_DIRS
debug3: Ignored env DBUS_SESSION_BUS_ADDRESS
debug3: Ignored env LESSOPEN
debug3: Ignored env WINDOWPATH
debug3: Ignored env DISPLAY
debug3: Ignored env LESSCLOSE
debug3: Ignored env XAUTHORITY
debug3: Ignored env COLORTERM
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug2: fd 3 setting TCP_NODELAY
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug2: channel 0: rcvd adjust 65536
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
Earl_Crowder
Trusted Contributor
Solution

Re: SSH Very Loaded

Check the size of /var/adm/wtmps, if it's large, and you don't need it, truncate it:
>/var/adm/wtmps

See this thread :


http://h30499.www3.hp.com/t5/System-Administration/SSH-Authentification-very-slow-on-HPUX-11-31/m-p/4361622#M346130

Mel Burslan
Honored Contributor

Re: SSH Very Loaded

I am assuming you do not get any delay before you enter the password and after entering the password, everything scrolls up fast until it hits the last line on this log. I that is right, your ssh is working fine whereas you system might be too busy to handle your shell request.
how is the system load ? and since you are suspecting too many ssh processes, check to see how many sshd processes are running at any given time by running

ps -ef | grep ssh | wc -l

at random intervals. Unless it is an out-of this world high number like in the order of thousands, your sshd is good for all intents and purposes and you need to look elsewhere for this problem
________________________________
UNIX because I majored in cryptology...
Bill Hassell
Honored Contributor

Re: SSH Very Loaded

What version of HP-UX are you running? Did you install the KRNG (Kernel Random Number Generator) product:

https://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=KRNG11I

Without this add-on, the random number generator code is based on several commands that can sometimes take a very long time to complete. The RNG is called during the key exchange during authentication.


Bill Hassell, sysadmin
AZayed
Super Advisor

Re: SSH Very Loaded

Hi Folks,
Hope you doing well.

Yes, maybe my SSH is working fine but my system is overloaded.

Now I grep for SSH and there is the output :

# uname -r
B.11.23

08:33 : Number of SSH is 23 & Number of sftp is 22
08:34 : Number of SSH is 2 & Number of sftp is 1
08:35 : Number of SSH is 6 & Number of sftp is 5
08:36 : Number of SSH is 2 & Number of sftp is 1
08:37 : Number of SSH is 6 & Number of sftp is 5
08:38 : Number of SSH is 22 & Number of sftp is 21
08:39 : Number of SSH is 6 & Number of sftp is 5
08:40 : Number of SSH is 10 & Number of sftp is 9
08:41 : Number of SSH is 2 & Number of sftp is 1
08:42 : Number of SSH is 6 & Number of sftp is 5
08:43 : Number of SSH is 22 & Number of sftp is 21
08:44 : Number of SSH is 6 & Number of sftp is 5
08:45 : Number of SSH is 3 & Number of sftp is 2
08:46 : Number of SSH is 2 & Number of sftp is 1
08:47 : Number of SSH is 7 & Number of sftp is 6
08:48 : Number of SSH is 22 & Number of sftp is 21
08:49 : Number of SSH is 6 & Number of sftp is 5
08:50 : Number of SSH is 6 & Number of sftp is 5
08:51 : Number of SSH is 2 & Number of sftp is 1
08:52 : Number of SSH is 6 & Number of sftp is 5
08:53 : Number of SSH is 22 & Number of sftp is 21
08:54 : Number of SSH is 10 & Number of sftp is 9
08:55 : Number of SSH is 2 & Number of sftp is 1
08:56 : Number of SSH is 2 & Number of sftp is 1
08:57 : Number of SSH is 6 & Number of sftp is 5
08:58 : Number of SSH is 22 & Number of sftp is 21
08:59 : Number of SSH is 6 & Number of sftp is 5
09:00 : Number of SSH is 10 & Number of sftp is 9
09:01 : Number of SSH is 2 & Number of sftp is 1
09:02 : Number of SSH is 2 & Number of sftp is 1
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
AZayed
Super Advisor

Re: SSH Very Loaded

Hi,
Also, this is the output for wtmpx

# ls -l /var/adm/w*
-rw-rw-rw- 1 adm adm 263520 Dec 24 02:45 /var/adm/wtmp
-rw-rw-r-- 1 adm adm 183060 Jun 15 2008 /var/adm/wtmp.Jun15-08
-rw-rw-rw- 1 adm adm 432815856 Jan 6 09:22 /var/adm/wtmps
-rw-rw-r-- 1 adm adm 6877184 Jun 15 2008 /var/adm/wtmps.Jun15-08
-rw-r--r-- 1 root root 280 Dec 24 02:45 /var/adm/wtmpx
#
# du -sk /var/adm/w*
264 /var/adm/wtmp
184 /var/adm/wtmp.Jun15-08
436704 /var/adm/wtmps
6728 /var/adm/wtmps.Jun15-08
8 /var/adm/wtmpx
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
AZayed
Super Advisor

Re: SSH Very Loaded

Also,
This is the OpenSSH version information. I think it's the latest one.

OpenSSH_5.6p1+sftpfilecontrol-v1.3-hpn13v7, OpenSSL 0.9.8o 01 Jun 2010
HP-UX Secure Shell-A.05.60.002, HP-UX Secure Shell version
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.
AZayed
Super Advisor

Re: SSH Very Loaded

Thanks folks,

SSH reads wtmps file and it seems I have a big file. around half gig.

Now I trim the file it's working very well.

/usr/sbin/acct/fwtmp < /var/adm/wtmps > /var/adm/wtmps-Jan-06-2011.ASCII

cat /dev/null > /var/adm/wtmps
Success seems to be connected with action. Successful people keep moving. They make mistakes, but they don't quit.