telnet hang

[Steve_3]

I had a problem with my K570 11.0 server
this morning. Running with about 4-5 oracle database. 6 processor and 3 Gig of Memory. When I try to telnet to it I got:

Trying....
Connected to server.domain.com
Escape character is '^]'

It stops right there. I try to login from
the console and could not get a login prompt.
I was able to ping the server.
I was reading the forum on "telnet" some suggestion was "nstrltel" kernel parm. The number of telnet session is set to 60 and the average telnet is about 12. Top...about 250-300 process running. Nproc kernel parm is set to 1200. No fork failure error message..

Any other suggestion why the telnet session hung?

thanks,
steve

[Barry O Flanagan]

Hmmm, if it was a nstrpty or nsrtrtel param set too low you might get something like "telnet device drivers missing" returned when you try to telnet in.

Could be a system resources problem - if you can get onto the box have a look around and go thru the usual top, sar and glance if you have it.

Could be a name resolution problem as well. Hows your nsswitch.conf set up?

[JACQUET]

What the average load time of your K ?
w | head -1

If too high, maybe your server could fork no more process...

PJA

[harry d brown jr]

From another session that is already logged in, can you get to a unix shell? There are a few reasons why you can't login, but the biggest one is to make sure you don't have a full filesystem, especially in VG00, or a missing mount point. If you can't get to a unix shell, and your system console is not responding to a login, will it respond to a "ctrl-b"? You will probably need to halt the system and reboot.

[Sridhar Bhaskarla]

There are a lot of reasons why the telnet hangs
if
-the bootable disk is bad
-there is any problem with anyone of the I/O components
-the kernel parameters overflow
-swap or memory overflow
-there is any problem with the network
-there is any problem with the DNS server or the internal resolve
-outdated PDCs

and lot many other reasons

I would try to find myself first. If I am unsuccessful, I will take a dump of the system and give to HP for analysis.

-Sri

[Steve_3]

here is the average load
11:15am up 3:04, 12 users, load average: 1.12, 1.15, 1.09

I had a session up at that time but there was no response on that session. I had to do a Ctrl-b and RS on to reboot. It is up and running fine now.

Maybe I need to tuned a certain kernel parameter.

Cound not telnet or rlogin. Don't think it is DNS or name resolution problem. Don't think it is vg00 b/c I have script if pass a certain threshold to page me.

thanks,
steve

[Sridhar Bhaskarla]

Steve,

Check the PDC version and upgrade it to the latest. Not happened on K5xx but I had the same problem with L-Series where HP suggested to upgrade the PDC. The problem went away.

-Sri

[Joseph Chakkery]

Hello,

I had a similar problem on my server. I could not login from console or telnet. I had to bounce the server. After restart I checked the OLD syslog and found an error message saying Proc table full. It seems some proces created somany child process and proc table was full.

May be u will have to monitor that.

Regards
Joe.

[MANOJ SRIVASTAVA]

Hi Steve

Do a sar -v 2 10 and check the o/p 's since it will give the current value to max value . Also check for nproc and maxuprc .

Manoj Srivastava

[Steve_3]

Here is the output from: sar -v 2 10

HP-UX dbs03 B.11.00 E 9000/899 08/20/01

11:56:20 text-sz ov proc-sz ov inod-sz ov file-sz ov
11:56:22 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:24 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:26 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:28 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:30 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:32 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:34 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:36 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:38 N/A N/A 314/1400 0 1668/1668 0 4173/10010 0
11:56:40 N/A N/A 313/1400 0 1668/1668 0 4166/10010 0


thanks,
steve

[Sridhar Bhaskarla]

Steve,

Please don't ever suspect inode table unless you have a lot of HFS file systems. Do you have anything written into syslog?. If you have MWA installed on the box, I would suggest to look at the history of various parameters like

CPU

GBL_CPU_TOTAL_UTIL
GBL_CPU_USER_MODE_UTIL
GBL_CPU_SYS_MODE_UTIL
GBL_PRI_QUEUE
GBL_RUN_QUEUE

DISK

GBL_DISK_UTIL_PEAK
GBL_DISK_SUBSYSTEM_QUEUE
GBL_MEM_CACHE_HIT_PCT

MEMORY

GBL_MEM_UTIL
GBL_MEM_PAGEOUT_RATE
GBL_MEM_SWAPOUT_RATE
GBL_MEM_QUEUE

NETWORK
GBL_NET_OUTQUEUE
GBL_NET_IN_ERROR_PCT
GBL_NET_COLLISION_PCT
GBL_NETWORK_SUBSYSTEM_QUEUE

KERNEL PARAMETERS
TBL_PROC_TABLE_UTIL
TBL_SHMEM_TABLE_UTIL
TBL_SEM_TABLE_UTIL
TBL_MSG_TABLE_UTIL
TBL_FILE_TABLE_UTIL
TBL_FILE_LOCK_UTIL

Other important thing I would monitor is
GBL_IPC_SUBSYSTEM_QUEUE

It will be good idea to enable sar and collect the information. Keep this in your crontab file so that you would know what were the resource utilizations during the system hang.

20 * * * * /usr/lbin/sa/sa1 1200 3

Check your default max_dbc_pct. If it is the default of 50%, it could be a problem. I found it causing the kernel to thrash and hang the system eventually.

-Sri

[Kevin Wright]

Could you ping the box? I had a situation a few months ago which the whole box was 'froze', couldn't telnet, console, ping or anything..happened two times before we realized it was the CPU's locking up all the busses. If it happens again, see if you can ping it, if not, you could have something locking up you coreI/O and other busses.

[Kevin Wright]

Sorry, I didn't read close enough, you could ping the box..but what port were you trying to telnet to..23? you should not get the escape character line on port 23, once your connected, it should spawn login..also you could not get a console so check inittab..you may have a getty problem.

[Barry O Flanagan]

Steve,

I wonder could it be a file descriptors problem? Whats nfile set to in the kernel?