vmunix: file: table is full --completely crashed server

[Ratzie]

This has happened twice now to our server, and I have not been able to pin point the problem.
When it does, it happens quick and starts core dumps.
1. I think I need to rearrange the kernel parameters but have difficult time figuring out what to change them to.
2. What can I do to find what is causing this. I have started to monitor the syslog for this error, but even if it does happen again, I would not know what to do except watch the system crash.

3. I was told I should increase maxusers up to 400, with this setting it will also bump up nproc,nfile,ninode.
It was also mentioned to raise to maxfiles to 500.

The biggest thing is to figure out why, and how to stop it next time!!!
Here are some particulars about the system:

Oracle 8.1.7.4
Model: 9000/800/L3000-7x
Main Memory: 8192 MB
Processors: 4
OS mode: 64 bit

Swap configuration

type size priority device/location
dev 4096 1

Kernel Configuration

The following drivers or parameters are configured into your system's
kernel. After installing HP-UX, use the sam(1m) command to configure
the following items into the kernel:
STRMSGSZ 65535
dbc_max_pct 10
maxdsiz 0X0C0000000
maxdsiz_64bit 0X0000000400000000
maxfiles 200
maxfiles_lim 2048
maxssiz 0X04FB3000
maxssiz_64bit 0X10000000
maxswapchunks 2048
maxtsiz 0X40000000
maxtsiz_64bit 0X100000000
maxuprc 1000
maxusers 250
maxvgs 80
msgmax 32768
msgmnb 65520
msgmni 664
msgseg 26560
msgssz 128
msgtql 256
nbuf 256
nflocks 2600
nproc 2020
npty 200
nstrpty 200
num_tachyon_adapters 5
semmni 10000
semmns 20000
semmnu 2016
semume 256
shmmax 0X40000000
shmmni 1024
shmseg 1024

[Joseph Loo]

hi,

as u may have seen in many post regarding this problem:

this error condition can usually be corrected by increasing the
value of NFILE and/or MAXUSERS in the kernel configuration file
and rebuilding the kernel. increasing the MAXUSERS parameter
will n turn increase the NFILE, NPROC, and NINODE parameters.

i do not see nfiles parameters.


another doc, HPUXKBRC00008909 mention about run sar command to check the file-sz column and also increasing MAXFILES:

http://www1.itrc.hp.com/service/cki/docDisplay.do?docLocale=en_US&docId=200000064128770

looks like MAXFILES of 200 is too small.

regards.

[vinod_25]

hi

This message indicates that your system has reached the defined limit of a kernel parameter.

To determine which parameter, use the convention of placing the letter "n" in front of the name in the table full message. For example, the following message on your console:

file: table is full

indicates that the "nfile" kernel parameter table size limit has been reached.

You can correct a kernel parameter limit situation in one of two ways:

- Decrease the use of the table. For instance, in the case of nfile, you could have users and applications close files that they do not need to have open. This is typically used as a short-term solution for the problem.

- Increase the kernel parameter that defines the limit on the table. You can increase the parameter using sam(1M). A system reboot is required to modify kernel parameters. By default, the kernel parameter "maxusers" is used in the calculation of several other parameters, and may be a better parameter to increase when system use is expanding. You would only want to modify maxusers, though, if doing so increases the parameter in question appropriately.

Regards

Vinod K

[Warren_9 · Accepted solution]

hi,

Beside monitor the syslog, you better turn on the sar to monitor the nfile usage.

"sar -v" will show the file-sz <-- nfile usage.

[Mahesh Kumar Malik]

Hi

You have to increase value of following parameters

NFILE
NPROC
NINODE

Easiast way to increase max no of users ( say by 50%), rebuilt the kernel and thus reboot

Regards
Mahesh

[Ratzie]

I have increased the size and no probs.
Thanks