topic Re: Wrong group membership in Operating System - HP-UX

Wrong group membership

Ricardo R Peirano — Thu, 25 Nov 2004 13:58:39 GMT

Hi everybody
I have one of my HP-UX 11iv1 servers with WebSphere on it and I have problems with the number of open files getting the nfile limit.
Do you know a way to know the number of open files for each process in the system?

Re: Wrong group membership

Asad Malik — Thu, 25 Nov 2004 14:20:04 GMT

you can use glance or lsof

Re: Wrong group membership

Slawomir Gora — Thu, 25 Nov 2004 14:37:45 GMT

Hi,

if you don't have glace you can use Q4 dump debugger to check number of used nfile.

1. preprocess the kernel for debugging

/usr/contrib/bin/q4pxdb /stand/vmunix

2. start Q4 using the running kernel
/usr/contrib/bin/q4 /stand/vmunix /dev/mem

3. in Q4 prompt run:

load struct file from file max nfile; keep f_count

4. result: kept XXX of YYY
XXX - files opened
YYY - nfile set in kernel

Re: Wrong group membership

Ricardo R Peirano — Thu, 25 Nov 2004 15:04:18 GMT

Guys
I have glance, but what I want to know is the number of open files that a particular process has opened. With this I hope to find the process which is causing this abrupt increase of opened files.
First I thougth to do this using lsof and an awk program but I want to know if there is any other way to do this thing.

Slawomir: your approach it's very good, IÂ´m tinking to write a script that will check the utilization of nfile and then recognize the process which is taking the major part of this tabl

Re: Wrong group membership

Bill Hassell — Thu, 25 Nov 2004 19:56:50 GMT

There is no way to summarize each process as to the number of open files. While there is a limit that you can impose on each process with a ulimit option. However, this would require adding ulimit to the startup environment for WebSphere and other applications.

However, the question is: why not increase nfile to 2 or 2 times it's current value? Knowing that some process in WebSphere or some other application needs 500 files open at the same time isn't something you can change (unless you can rewrite the code and recompile). Note that nfile is way too small in HP-UX for anything but a simple 1-user workstation. It is not unusual to have nfile=4000, even nfile=50000 is not a problem.

Re: Wrong group membership

Ermin Borovac — Thu, 25 Nov 2004 20:09:00 GMT

With glance you can see open files for a particular process as follows

$ glance -F
Enter PID (14227) :

There doesn't seem to be an easy way to get per-process file count in glance.

Here is lsof/perl one-liner to achieve this. It should print output 'PID: ' and it should be sorted by number of open files (highest usage processes listed at the end).

lsof -d 0-10000 2>/dev/null | perl -ane '!/PID/ && $count{$F[1]}++; END {for (sort {$count{$a} <=> $count{$b}} keys %count) {print "$_: $count{$_}\n"}}'

Re: Wrong group membership

F Verschuren — Fri, 26 Nov 2004 04:00:55 GMT

you can use lsof (list of all open files)(freeware I think) If you grep on the proces ID you see all the open files

Re: Wrong group membership

Ricardo R Peirano — Fri, 26 Nov 2004 11:42:37 GMT

I wrote down a script as follows to catch the WebSphere App that is keepening a lot of file opened.

ps -efx | grep native | while read line
do
line1="`echo $line | awk -F\"ActiveEJBServerProcess\" '{ print $1}'"
line2="`echo $line | awk -F\"ActiveEJBServerProcess\" '{ print $2}'"

pid="`echo $line1 | awk '{ print $2 }'`"

openfiles="`/usr/contrib/bin/lsof -p $pid | wc -l `"

echo $line2 | awk -F: '{ print $2 }' | sed -e "/^$/d" | awk -v p="$pid" -v f=$openfiles '{ print p" "$
1"\t"f" archivos abiertos" }'

done

Now, all is working well but I'l trying to take pictures until the problem arises, and then recognized wich app is failing.

Regards, Ricardo.

Re: Wrong group membership

Ricardo R Peirano — Fri, 26 Nov 2004 11:48:50 GMT

I wrote down a script as follows to catch the WebSphere App that is keepening a lot of file opened.

ps -efx | grep native | while read line
do
line1="`echo $line | awk -F\"ActiveEJBServerProcess\" '{ print $1}'"
line2="`echo $line | awk -F\"ActiveEJBServerProcess\" '{ print $2}'"

pid="`echo $line1 | awk '{ print $2 }'`"

openfiles="`/usr/contrib/bin/lsof -p $pid | wc -l `"

echo $line2 | awk -F: '{ print $2 }' | sed -e "/^$/d" | awk -v p="$pid" -v f=$openfiles '{ print p" "$
1"\t"f" archivos abiertos" }'

done

Now, all is working well but I'll trying to take pictures until the problem arises, and then recognized which app is failing.

Regards, Ricardo.