topic Re: nfile is maxing out in Operating System - HP-UX

nfile is maxing out

joe clark — Thu, 21 Feb 2002 20:29:46 GMT

We have a nbox with 11i, around 1 am the nfile table is filling up, we have sar -v's and ps -ef,I had up'd nfile from 8000 to 16000, so now it took 45 minutes from 1 am to fill up, tonight I am going to use glance to see what process is opening all those files, any other ideas with glance, or other ways to catch what is filling up the nfile table.
nproc is fine it doesn't grow at all.
Thanks,
Joe

Re: nfile is maxing out

joe clark — Thu, 21 Feb 2002 20:31:34 GMT

The nfile table grows from 4000 to 16000 in that 45 minutes.

Re: nfile is maxing out

A. Clay Stephenson — Thu, 21 Feb 2002 20:36:25 GMT

Hi Joe:

I would download a copy of lsof from any of the HP-UX Porting Centre's. It is really the weapon of choice in this case. If will list all the open files and the process that has them open - just what you need. Because you say this is happening as 0100, my pschic, Miss Cleo, tells me to look for backup programs. She also suggests that you do a crontab -l and look for batch jobs.

Regards, Clay

Re: nfile is maxing out

Alan Riggs — Thu, 21 Feb 2002 20:42:03 GMT

Clay is right, lsof is the tool you need. You can generate a list of open files by process, then use awk (or cut|uniq -c) to generate a count of the number of files opened by each process.

It shows all the signs of a process spinning out of control. It might be backup related. I have also seen strange recursion in mail or print queues causing this kind of symptom, but lsof will nail down the culprit for you.

Re: nfile is maxing out

Helen French — Thu, 21 Feb 2002 20:48:20 GMT

Hi,

Clay and Alan are right. lsof is a good tool for find out the root cause of the problem.

Also check for any cron jobs running at the specific time (crontab -l). Any other processes or applications running at the specific time ( check all log files).

Check here: http://hpux.asknet.de/hppd/hpux/Sysadmin/lsof-4.55/

HTH,
Shiju

Re: nfile is maxing out

Sanjay_6 — Thu, 21 Feb 2002 20:48:56 GMT

Hi,

Download and install lsof, this might help you,

lsof for 10.20 and 11

http://hpux.connect.org.uk/hppd/hpux/Sysadmin/lsof-4.55/

lsof for 11i,

http://www.mirrors.wiretapped.net/security/host-security/lsof/binaries/hpux/B.11.11/

Hope this helps.

Regds

Re: nfile is maxing out

T. M. Louah — Thu, 21 Feb 2002 23:00:09 GMT

To add more to above:

To sort all the proceses by memory size:
# UNIX95= ps -eo vsz,ruser,args | sort -rn | more

If any of the processes are pushing more than 50 megs, run SAM & change maxdsiz to double or even more (requires a reboot).

.. if sar -v 5 5 get high values on proc-sz, inod-sz & file-sz u might want to increase maxusers the following is the impact of maxusers value :

root:> more /stand/build/tune.h |grep -i maxuser
#define MAXUSERS any_value
#define NCLIST (100+16*MAXUSERS)
#define NFILE (16*(NPROC+16+MAXUSERS)/10+32+2*(NPTY+NSTRPTY+NSTRTEL))
#define NINODE ((NPROC+16+MAXUSERS)+32+(2*NPTY))
#define NPROC (20+8*MAXUSERS)

g`d luck
t+-