open files - lsof

[Joseph Hoh]

We are having an issue with the number of open files on our server. I can increase the nfile parameter but I am afraid that this will not solve the problem. I believe the nfile usage will just continue to increase untill it hits 100%.

Can anyone tell me how to compare the output of lsof to the nfile usage I see in Glance. I would like to try to tie together the two outputs to determine who/what is opening all of these files.

thanks

[Michael Steele_2]

The current 'nfile' value as well as its ceiling can be displayed with 'sar -v'. If you divide the numerator into the denominator you'll get the percentage used of the system file table.

'lsof' has no relation to 'nfile' since 'lsof' is not a systemwide metric and 'nfile' is kernel specific.

If you read the man page listed below most of 'lsof's usage is file system specific.

Here are some examples:

lsof -p pid
lsof -u uid
lsof -i tcp:80
lsof /file_system

[Michael Steele_2]

man page

http://hpux.cict.fr/hppd/hpux/Sysadmin/lsof-4.64/man.html

[Bill Hassell]

nfile *must* be increased if you continue to add additional users and processes. There is nothing wrong with that requirement, but do NOT be stingy when you increase this parameter. If it is sitting at 500, then increase it to 2000. The minimum should be double (unless it's OK to change the kernel and reboot a lot). It's not unusual to set nfile to 10,000, 50,000 or even 500,000. Remember that *every* program starts with 3 files (stdin, stdout and stderr) and usually opens a few more. So a typical rule of thumb is nproc*5 assuming most processes are 'normal'. There can always be a (questionable design) program that requires 900 files to be opened at the same time.

Glance will show all the files for a selected process but does not have the ability to sort processes by the number of open files.

Another limit to prevent runaway programs from using too many files is to set ulimit -n in /etc/profile (see man sh-posix) or reduce the value for maxfiles in the kernel. maxfiles can be overridden programatically with setrlimit but for casual programs, this is a way to reduce the max files opened per program (default is 60).

[V.Tamilvanan]

Hi,
lsof gives the output based on pid/Directory/File/Filesystem/Ip address. It doesnot give you system wide open file status.
The better way of checking usage on nfile is

#sar -v 3 3

the output look like the below. In that file-sz column represents 517 open files are there out of maximum 1899.

21:17:04 text-sz ov proc-sz ov inod-sz ov file-sz ov
21:17:07 N/A N/A 131/820 0 560/1088 0 517/1899 0
21:17:10 N/A N/A 131/820 0 560/1088 0 517/1899 0
21:17:13 N/A N/A 131/820 0 560/1088 0 517/1899 0

HTH