System Panic

[Senthilkumaran.T]

Hi,

One of OUR K-460 Class Server running on HPUX 10.2 panic with the following error message.

inetd[704]: netbios_ssn/tcp: accept: File table overflow
vmunix: file: table is full

After system panic, The server was unable to reboot and unable to telnet to the server also. The only option left aws to HardBoot the system.

Could you tell me what is wrong with this system and How we can prevent it to be happened again.

Thanks
Senthil

[linuxfan · Accepted solution]

Hi Senthil,


Looks like you ran out of nfiles. You might have to increase the kernel parameter nfile. nfile parameters determines the maximum number of files that can be open at any one time, system wide.

For information about kernel parameters look at

http://docs.hp.com/hpux/onlinedocs/os/11i/kcparams/KCparams.OverviewAll.html

-Regards
Ramesh

[Thierry Poels_1]

hi,

your nfile kernel parameter is probably too low. Check its setting with "sar -v 4 4" and increase it if necessary.

good luck,
Thierry.

[Magdi KAMAL]

Hi,

It seems like your kernel parameter nfile "max number of open files " is small.

Try configuring a greater value of what you have already now.

Magdi

[James R. Ferguson]

Hi:

You need to increase the kernel parameter 'nfile'. This defines the maximum number of files that the system can have open simultaneously.

Since this parameter is dependent upon several others, including 'maxusers', go to SAM -> Kernel Configuration and increase the number of 'maxusers' which will inflate 'nfile'.

To monitor 'nfile' you can use 'glance' (toggle its "t"able view) or you can use 'sar' to see the kernel ceiling, the current value and the number of times the table has overflowed.

# sar -v 5 10

...will report 5-samples at 10-second intervals. Look at the column "file-sz".

To better understand the kernel parameters take a look at this document:

http://docs.hp.com/hpux/onlinedocs/os/KCparams.OverviewAll.html

Regards!

...JRF...

[eran maor]

Hi

you need to increase the nfile kernel param to solve problem of File table overflow .

i think that the service netbios_ssn ( you can see the number of port in the /etc/services
had a socket open and it open a lot of file .

you can try to see in with glance .

but increasing the nfile will solve the problem

[Senthilkumaran.T]

Hi ,
The following output is generated from sar -v 4 4

21:05:53 text-sz ov proc-sz ov inod-sz ov file-sz ov
21:05:57 N/A N/A 246/700 0 1520/1520 0 786/1632 0
21:06:01 N/A N/A 257/700 0 1520/1520 0 803/1632 0
21:06:05 N/A N/A 256/700 0 1520/1520 0 799/1632 0
21:06:09 N/A N/A 256/700 0 1520/1520 0 799/1632 0

Are they any problems on indoe? Do I have to increas the I node also?. How much I need to increase (Approximately)?

Thanks
Senthil

[Magdi KAMAL]

Hi again,

Your are reaching the max inode size 1520/1520 !!!


If your are running hp-ux 11.0, have a look on the following thread ( from www.docs.hp.com )explaining ninode to up to 15000 !


http://www.docs.hp.com/hpux/onlinedocs/os/11.0/tuningwp.html#ninode

Magdi

[Magdi KAMAL]

If your are running 10.20, you can increase the ninode but not more than 4000 due to the spinlock contention problem.

Magdi

[linuxfan]

Hi Senthil,

On a JFS filesystem, there is no inode limit as it dynamically allocates inodes. But if you have a lot of HFS filesystems then you need to worry about ninode parameter.

Also a 100% usage of ninodes does not indicate a problem, but if you want its safe to increase it to 4000.

Here is a link which might be of interest.
http://forums.itrc.hp.com/cm/QuestionAnswer/1,1150,0x5cf7d211e18ad5118ff10090279cd0f9,00.html

-HTH
Ramesh

[Thierry Poels_1]

Hi,

the two most important things you want to monitor with "sar -v" are 'proc-sz' and 'file-sz' which are kernel parameters 'nproc' and 'nfile'. They define the maximum number of processes on your system and the maximum number of open files on your system. If you hit this limits you're in trouble, system panic for example. You should monitor them regularly.

The 'inod-sz' is of little importance if you are using VXFS filesystems (as already mentioned).

good luck,
Thierry