maxfiles_lim reached, which files keeps application open ?

[Rob van Buiten]

On our hp-ux v11.31 server a java application (3th party) keeps "files" open, so maxfiles_lim limit is reached.
Workaround is application stop and start, but that has great impact on production.
So, I want to find out what is kept open.
I have installed and used lsof, but output is telling me little.


regards,
Paul Ettema.

[Steven E. Protter]

Shalom Paul,

You have already found the problem.

maxfils_lim, nfiles, maxuprc need to be increased on your system.

Use the SMH interface, increase them to perhaps double their current limits and see how long your system rusn before hitting the wall.

The java application may need vendor support to close files when they don't need to be open. This could also be a function of java patches which can be found at http://www.hp.com/go/java A java update might help as well.

With regards to lsof, which is the best tool, some things to try:

http://sial.org/howto/debug/unix/lsof/

http://www.ibm.com/developerworks/aix/library/au-lsof.html

Lots of good examples in there.

SEP

[Mel Burslan]

If you don't mind, let me answer your question with another question:

What are you going to do when you find out which files are being kept open ? If the third party application is keeping them open, you can not go willy-nilly and kill the processes which keep them open. Otherwise you are jeopardizing the application health.

What are you trying to accomplish here ? If you know that your third party java application is not doing a good job of housekeeping, you should contact the vendor and have them clean up their code.

Otherwise, you might consider increasing the maxfiles and maxfiles_lim kernel parameters to a higher value but this requirement needs to be assessed with caution as too high of a number here can adversely impact your overall system performance.

Hope this helps

[James R. Ferguson]

Hi Paul:

I agree with Mel. The _requirements_ of the application may mandate a larger 'maxfiles_lim'. I would consult the application vendor for _their_ recommended kernel settings.

Regards!

...JRF...

[Dennis Handly]

From your output, there seems to be lots of sockets and even more FIFOs.

You may want to use tusc to trace opens/closes but they will tell you the same thing, 3000+ files open.

But perhaps armed with the list of open files the vendor can tell you if this is normal or a bug.

[Rob van Buiten]

@all (Steven, Mel, James)

Some more info here:

the java appl. don't close open files when it do not used these. So in a while after 20 days maxfiles_lim of 4096 is reached. so incresing won't help.

What I (and maker off appl) want to know, what is going wrong, what is left open.
So that the maker can solf the bug.

The problem is in the application not the value of maxfile_lim.

[Rob van Buiten]

@Dennis

Where can I find "tusc" I want to try.
Not the ammount is now important, but what.

Paul Ettema.

[Dennis Handly]

>What I (and maker off appl) want to know, what is going wrong, what is left open.

(You're scaring me. ;-)
The application maker should know how his application works. There aren't that many files to keep track of. Perhaps a tusc trace may give you ideas.

[James R. Ferguson]

Hi (again):

> Where can I find "tusc" I want to try.

http://hpux.connect.org.uk/hppd/hpux/Sysadmin/tusc-7.10/

Regards!

...JRF...

[Rob van Buiten]

Thanks, James.
Found it also 1 minute ago.
Thanks.

Paul.