System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

maxfiles_lim reached, which files keeps application open ?

Rob van Buiten
Regular Advisor

maxfiles_lim reached, which files keeps application open ?

On our hp-ux v11.31 server a java application (3th party) keeps "files" open, so maxfiles_lim limit is reached.
Workaround is application stop and start, but that has great impact on production.
So, I want to find out what is kept open.
I have installed and used lsof, but output is telling me little.


regards,
Paul Ettema.
11 REPLIES
Steven E. Protter
Exalted Contributor

Re: maxfiles_lim reached, which files keeps application open ?

Shalom Paul,

You have already found the problem.

maxfils_lim, nfiles, maxuprc need to be increased on your system.

Use the SMH interface, increase them to perhaps double their current limits and see how long your system rusn before hitting the wall.

The java application may need vendor support to close files when they don't need to be open. This could also be a function of java patches which can be found at http://www.hp.com/go/java A java update might help as well.

With regards to lsof, which is the best tool, some things to try:

http://sial.org/howto/debug/unix/lsof/

http://www.ibm.com/developerworks/aix/library/au-lsof.html

Lots of good examples in there.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
Mel Burslan
Honored Contributor

Re: maxfiles_lim reached, which files keeps application open ?

If you don't mind, let me answer your question with another question:

What are you going to do when you find out which files are being kept open ? If the third party application is keeping them open, you can not go willy-nilly and kill the processes which keep them open. Otherwise you are jeopardizing the application health.

What are you trying to accomplish here ? If you know that your third party java application is not doing a good job of housekeeping, you should contact the vendor and have them clean up their code.

Otherwise, you might consider increasing the maxfiles and maxfiles_lim kernel parameters to a higher value but this requirement needs to be assessed with caution as too high of a number here can adversely impact your overall system performance.

Hope this helps
________________________________
UNIX because I majored in cryptology...
James R. Ferguson
Acclaimed Contributor

Re: maxfiles_lim reached, which files keeps application open ?

Hi Paul:

I agree with Mel. The _requirements_ of the application may mandate a larger 'maxfiles_lim'. I would consult the application vendor for _their_ recommended kernel settings.

Regards!

...JRF...
Dennis Handly
Acclaimed Contributor

Re: maxfiles_lim reached, which files keeps application open ?

From your output, there seems to be lots of sockets and even more FIFOs.

You may want to use tusc to trace opens/closes but they will tell you the same thing, 3000+ files open.

But perhaps armed with the list of open files the vendor can tell you if this is normal or a bug.
Rob van Buiten
Regular Advisor

Re: maxfiles_lim reached, which files keeps application open ?

@all (Steven, Mel, James)

Some more info here:

the java appl. don't close open files when it do not used these. So in a while after 20 days maxfiles_lim of 4096 is reached. so incresing won't help.

What I (and maker off appl) want to know, what is going wrong, what is left open.
So that the maker can solf the bug.

The problem is in the application not the value of maxfile_lim.

Rob van Buiten
Regular Advisor

Re: maxfiles_lim reached, which files keeps application open ?

@Dennis

Where can I find "tusc" I want to try.
Not the ammount is now important, but what.

Paul Ettema.
Dennis Handly
Acclaimed Contributor

Re: maxfiles_lim reached, which files keeps application open ?

>What I (and maker off appl) want to know, what is going wrong, what is left open.

(You're scaring me. ;-)
The application maker should know how his application works. There aren't that many files to keep track of. Perhaps a tusc trace may give you ideas.
James R. Ferguson
Acclaimed Contributor

Re: maxfiles_lim reached, which files keeps application open ?

Hi (again):

> Where can I find "tusc" I want to try.

http://hpux.connect.org.uk/hppd/hpux/Sysadmin/tusc-7.10/

Regards!

...JRF...
Rob van Buiten
Regular Advisor

Re: maxfiles_lim reached, which files keeps application open ?

Thanks, James.
Found it also 1 minute ago.
Thanks.

Paul.
Dennis Handly
Acclaimed Contributor

Re: maxfiles_lim reached, which files keeps application open ?

>Where can I find "tusc" I want to try.
>Not the amount is now important, but what.

You have the "what" with your lsof output. If you want where, you would need to debug, perhaps with a breakpoint in open or socket.
See these old links to download:
http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=1311146
http://forums.itrc.hp.com/service/forums/questionanswer.do?threadId=990612
Rob van Buiten
Regular Advisor

Re: maxfiles_lim reached, which files keeps application open ?

@all.

tusc installation: simple
testing tusc (on pre-production) no problems.

first run on production: bad. :-(
errors on screen and application has created a child and not function anymore.
we have to restart application.

So, investigation is on hold.

Thnx for reply

Paul Ettema.