HPUX java 1.4.2 NIO too many open files

[Marco Ullasci]

I've a program written in java running on HPUX 11i with SDK 1.4.2_08. It uses NIO.
The program is basically a gateway with a socket interface to query an ejb object.
Under heavy load it starts to get problems with the IO and reports "too many open files".
I've noticed a large number of socket in FIN_WAIT_2 state on the application side and CLOSE_WAIT on the client side.
I've tried to have the number of open files reduced enabling a timeout for FIN_WAIT_2 using ndd.
This worked at the "netstat -an" level, but I still get the problem.
I've checked with glance what happens inside the process and I've found more than 2000 lines like this one:
2k socket rd/wr 1 59

Anyone can help me?

Thanks
Marco

[Marco Ullasci]

I'll add that I noticed the following behavior:
According to Glance sockets are generated as:
nnn inet,tcp socket rd/wr
Then when they're closed they go in FIN_WAIT_2 from ESTABLISHED.
After 90s (the configured timeout) they disappear from the "netstat -an" radar and become, again according to glance, like this:
2k socket rd/wr 1 59
And are never released.

Thanks
Marco

[Marco Ullasci]

Additional information:
Using java 1.5

[version]
java version "1.5.0"
Java(TM) 2 Runtime Environment, Standard Edition (build 1.5.0-_24_nov_2004_16_29)
Java HotSpot(TM) Server VM (build 1.5.0 FCS (JS B64) jinteg:11.24.04-15:51 PA2.0 (aCC_AP), mixed mode)
[/version]

The problem remains exactly the same.

Thanks
Marco

[Alexey Roytman]

"too many open files" does not relate to a system socket in any state (waiting or not). It means that your application has many open file descriptors (sockets).

Try to find out *when* your leak handles/descriptors. (It's like memory leak fixing.)
Take a snapshot of fuser/lsof for your process. Connect and disconnect. Take new snapshot. Compare snapshots. Check file handles open.

Currently, that's all.