Getting "vmunix: file: table is full" error in syslog.log !!!

[rveri]

Hi All ,

I am getting "file table is full" error message.

"Aug 9 14:56:16 sapd002 vmunix: file: table is full
Aug 9 14:56:16 sapd002 above message repeats 1699 times
Aug 9 14:56:17 sapd002 vmunix: file: table is full
Aug 9 15:00:30 sapd002 vmunix: file: table is full
Aug 9 15:00:31 sapd002 above message repeats 904 times
Aug 9 15:00:32 sapd002 vmunix: file: table is full
Aug 9 15:01:26 sapd002 vmunix: file: table is full
Aug 9 15:01:29 sapd002 above message repeats 12 times
Aug 9 15:01:29 sapd002 syslogd: going down on signal 15
----------------

I was not able to run any command, and need to reboot the system. After reboot its fine.
But I believe this is because of kernle parameter problem.

I want to change the follwoing kernel values:
to this values:

nfiles=12318 Currently (8212)
ninode=6000 Currently (4000)
nproc=3620 Currently (1628)
maxfiles=2048 currently (200)
---------------------------------------

1) I am not able to change through SAM.
Can any one help in giving good steps (command prompt), and suggestions.
And any impact will be there.
(Server is N4000-55/ 8 CPU /16GBRAM. hpux 11.11. )

Tahnk you,

R.Veri.
------

[Mel Burslan]

here is your manual kernel compile steps:

cd /stand/build
/usr/lbin/sysadm/system_prep -v -s system
vi system # make changes here

mk_kernel -s system

mv /stand/system /stand/system.prev
mv /stand/vmunix /stand/vmunix.prev
mv /stand/dlkm /stand/dlkm.prev
mv /stand/build/system /stand

kmupdate /stand/build/vmunix_test
cd /
shutdown -r

[rveri]

Hi Mel ,

dlkm is a directory , does it needs to move also.

here is the output:
-----------------------------
$ ls -l
total 141312
-rw-r--r-- 1 root sys 19 Apr 24 2002 bootconf
drwxr-xr-x 5 root sys 3072 May 25 23:17 build
drwxr-xr-x 6 root sys 1024 Mar 27 09:41 dlkm
drwxrwxr-x 5 root root 1024 Feb 2 2005 dlkm.vmunix.prev
-rw-r--r-- 1 root sys 20392 Aug 9 15:13 ioconfig
-r--r--r-- 1 root sys 82 Mar 8 2002 kernrel
drwxr-xr-x 2 root sys 1024 Aug 9 17:15 krs
drwxr-xr-x 2 root root 1024 Aug 9 15:11 krs_lkg
drwxr-xr-x 2 root root 1024 Aug 9 17:15 krs_tmp
drwxr-xr-x 2 root root 8192 Mar 8 2002 lost+found
-rw------- 1 root root 12 Aug 9 15:11 rootconf
-rw-r--r-- 1 root sys 1812 Aug 9 16:46 system
drwxr-xr-x 2 root sys 1024 Mar 27 09:41 system.d
-rw-r--r-- 1 root sys 1812 Aug 9 15:31 system.old
-rw-r--r-- 1 root root 1807 May 25 22:36 system.prev
-rw-r--r-- 1 root sys 1793 May 25 22:35 system.prev.old
-rwxr-xr-x 1 root sys 24075312 Mar 27 09:12 vmunix
-rwxr-xr-x 1 root sys 24075312 Aug 9 15:23 vmunix.old
-rwxr-xr-x 1 root sys 24075312 May 25 21:15 vmunix.prev
$
-------------------------

Thanks ,
Veri.

[Cesare Salvioni]

hi

just a hint, be very carefully modifying kernel parameters by hand, some of them are connected together and there are rules to follow between the value.
If you try to override these rules your result could be a unbootable kernel. Not a big problem if you copy the running one in vmunix.prev and reboot with it but annoying and time loosing

I suggest to further investigate why SAM wan't change them. It is a real sam problem or sam is checking the values and avoid building a not runnable kernel? In this last case doing by hand is not the solution.

If this is the case SAM will tell you which parameter is conflicting, suggesting a limit value, sometimes you only must change the params in the right order to let SAM accept them.

Last thing, in my experience the value for nfiles (the param causing the log in syslog) you are choosing is really high, are you sure you need them? I mean there could be some process opening files in a wrong way or something other not working good. If this is the case raising the parameter will only move the problem. Did you change something on the host? did you installed something new? did you updated some package? did you check for new patches?
I would try to investigate further who is using the files before or after the change

hope it help

[Joseph Loo · Accepted solution]

hi,

that message means the maximum number of open files is exceeded.

your current values though shows your system is working reasonably well but the message will only shows when the open files r exceeded. a good indication to tell when the message appears again:

# sar -v 3 10
from the file-sz column, u will notice that the current is more than the nfile parameter.

refer to this doc:

http://www2.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000080094708

any reasons why u r not able to change thru SAM. r u not using root?

regards.
(p.s. please remember to assign points.)

[Devender Khatana]

Hi Veri,

You are not required to copy dlkm directory anywhere.

Follow this to achibe your changes. Also share the error encountered while doing it through SAM.

Change the directory to /stand/build.
#cd /stand/build
Create the kernel parameter system file from the running system.
#/usr/lbin/sysadm/system_prep -s system
Edit the system file to make necessary changes.
#vi system
Build the new kernel using the mk_kernel command.
#/usr/sbin/mk_kernel -s ./system
Back up the old kernel and system files.
#mv /stand/system /stand/system.old
#mv /stand/vmunix /stand/vmunix.old
Install the new kernel.
#cp /stand/build/system /stand
#cp /stand/build/vmunix_test /stand/vmunix
Reboot the system.
#shutdown -r 0
Verify the changed parameters and additional devices using the following commands.
#sysdef
#ioscan


HTH,
Devender

[Rajesh SB]

Hi,

The parameters nfile,ninode,nproc to be modified uses following formulas.

nfile (19*(NPROC+16+MAXUSERS)/10+32+2*(NPTY+NSTRPTY+NSTRTEL))
nproc (20+28*MAXUSERS)
ninode ((NPROC+16+MAXUSERS)+32+(2*NPTY))

In this case, SAM wouldn't allow to tune these parameters, until dependent kernel parameter like MAXUSERS, NPTY set to minimum SAM adviced value.

Are you facing this kind of problem? If so, it helps you.

Regards,
Rajesh