Open files for WebLogic Server 8.1

[Michael Janssens]

We installed a WebLogic Server 8.1 for PeopleSoft. Unfortunately I often reach error messages that the webserver "Too many open files". When I check with glance the Open files for the process, I see the number always increasing. But a lot of these files have the < unknown socket type >. Looks like none of the sockets are removed. Is there a way to perform this? Maybe with ndd configuration?

[Dave Hutton]

What does glance -t show?
System Table Available Used Utilization High(%)
--------------------------------------------------------------------------------
Proc Table (nproc) 2068 254 12 12
File Table (nfile) 4106 1899 46 46

I don't think it's sockets that your having issues with. I personally think it's number of files open.

[Michael Janssens]

Dave,

The value of glance -t is the following:
Proc Table (nproc) 4096 - 228 - 6 - 6
File Table (nfile) 63498 - 7981 - 13 - 13

For me they still look ok. But when I check number of open files in Glance -F and the process id of my WebServer I already have 126 pages full of open files. Of which the most are labeled .
This number is always increasing and I do not have that much users.

When I check the error code on BEA they suggest to increase maxfiles and maxfiles_lim, currently set to 60 and 2048. Also umilit -n is set to 2048.

But after a few days I feel this process is reaching this limit. Just because none of the open files are removed.

Just increasing does not look as a right solution as other limits will be reached.

Any suggestions of what there is more to monitor?

[Michael Janssens]

Today again, I reached the number of open files for the webserver and everything was blocked again.

The following error was received in the error log of BEA: BEA-000204 Failed to listen on port 443, failure count: 1; failing for 0 seconds, java.net.SocketExcpetion: Too many open files.

Does anyone see a away to remove these open files instead of stop and re-start of the webserver process?

[harry d brown jr]

my weblogic servers have

maxfiles 2048 - 2048
maxfiles_lim 2048 Y 2048
nfile 51548 - 51548


live free or die
harry d brown jr

[Michael Janssens]

Harry,

These values are almost the same as mine. One small question:

Can you check on glance the Open files for the WebLogic server (java) proces? Are you also getting these files with ?

[harry d brown jr]

unfortunately I didn't have it running, thuss when I did start it it only had about 60 files (a few sockets - like two or three).

Which service pack are you running and what is the patch level of your HP box??

live free or die
harry d brown jr

[Michael Janssens]

For HPUX 11i all the June 2004 patches have been applied. And the WebLogic 8.1 has SP3 applied.

I really would like to know were all these come from and why they are not removed from my system?

[Michael Janssens]

More info from the lsof tool.

When checking the open files with lsof I am receiving more information of these open files I get with the WebLogic process.

lsof -l provides the following output for the java process of WebLogic:

Command: java
PID: 27778
USER: 5000
FD: 240u
TYPE: STSO
Device: 21,0x0940051
SIZE/OFF: 0t11672
Node: 109
Name: /dev/tcp->tcpm->ip

Does anyone know this type STSO, and the link with /dev/tcp?

Is there not a ndd setting I need to add?
It really are this types of files that are always increasing for this process and I would like to reduce this.

[Michael Janssens]

A first suggestion was to put the following ndd configuration:
ndd -set tcp_time_wait_interval 60000
By this shorting the TCP wait time, and recycle the sockets more efficiently.

Currently I do not see the number dropping of these STSO files: Stream Sockets?

[Michael Janssens]

Setting the tcp_time_wait_interval did not resolve the problem. The number of Open files for the java process of WebLogic is currently 644 and 523 of them are linked to the Stream Sockets of /dev/tcp->tcpm->ip.
(see enclosed file)
There is noting I can do to resolve this.

Could it be that this due to the usage of MS Active Directory server?

[Michael Janssens]

A socket leak

This problem is beter known as a socket leak, for WebLogic 8.1,it happen when using https on the SP2 version. This problem is described on BEA as CR133071.

SP3 will resolve this.

I will install SP3 and inform you of the results.

[Michael Janssens]

Installing the weblogic patch resolved the problem. Thanks for the HP people who helped us in this matter.

[Michael Janssens]

Like written previously, the problem was know by weblogic and installing the patch resolved it.