Hp UX server problem

[Sandeep Geet]

Hi !

I have HP rp7410 server with HP UX 11i v1 with Informix 9.40.FC2 DB and Baan ERP.

I have encountered server crash, I have observed following error messages in syslog.
==============================================

Jan 17 14:43:24 MANTHAN vmunix: file: table is full
Jan 17 14:44:15 MANTHAN su: + tc root-informix
Jan 17 14:44:03 MANTHAN vmunix: file: table is full
Jan 17 14:44:22 MANTHAN above message repeats 58 times
Jan 17 14:44:22 MANTHAN rexecd[15386]: read: Connection refused
Jan 17 14:44:23 MANTHAN vmunix: file: table is full
Jan 17 14:46:21 MANTHAN vmunix: file: table is full
Jan 17 14:48:34 MANTHAN above message repeats 47 times
Jan 17 14:52:16 MANTHAN rexecd[15649]: PAM Status - 9, PAM Error Message - Authe
ntication failed
==============================================

Can someone help about what is file table full and how to monitor the same?

Thanks and Regards,
Sandeep

[Florian Heigl (new acc)]

it is the kernel parameter nfile, which holds the maximum number of open files.
'files' as in 'sockets, devices, named pipes and files' ;)

please use the following command to check it's current value and raise it a bit.
# kmtune -q nfile
Parameter Current Dyn Planned Module Version
===============================================================================
nfile 78848 - (15*NPROC+2048)

(Use SAM -> Kernel Configuration -> Configurable Parameters)

to be honest, I can't say how to query the average / peak values on Your system.
Right now they've been to low. :)

I hope someone fills in on that part.

[Robert-Jan Goossens]

Hi Sandeep,

Kernel parameters nfile nproc ninode.

Could you post the last part of your /stand/system file (after the line * Tunable parameters) and a sar -v 2 5 output ?

Regards,
Robert-Jan

[Rita C Workman]

As Florian well stated the issue you see is because your nfile parm is too low.
A good thing to watch for # of processes running, inodes being used and yes files is:
sar -v 1 60

This will show you active-number/parm-setting, and is helpful for tuning these.
Remember, tuning is not only increasing what you need, but decreasing what you don't. So watch the inode value, most folks have that set way to high when 2048 or 4096 often is sufficient.
But you decide....

Rgrds,
Rita

[Ravi_8]

Hi,

If you get vmunix: file: table is full, you need to tweak the nfile kernel parameter (number of files open at once).

[HGN]

Hi

This error points to kernel parameter tuning requirement, please make changes to Kernel parameters nfile ninode nproc.

This should take care of the problem.

Rgds

HGN

[Kent Ostby]

Sandeep --

Did the sysetm reboot itself or did you reboot it because you couldn't do anything on it ?

Did you use the "TC" command or button or just power cycle it ?

If it rebooted itself or you TC-ed it, it should generate a dump.

If you have a dump, you would also have an entry in /etc/shutdownlog. If the box rebooted itself then the message in /etc/shutdownlog may tell us more about what is going on.

If you simply power cycled it then you'll have to work the file table is full the way others have suggested.

If you had a dump in /var/adm/crash then we could also take a look to see which processes were filling up the file table and perhaps fix the root problem rather then just fix the symptom.

Best regards,

Oz