HPUX behavior with insufficient kernel settings

[Tim Killinger]

Hello,

Help! We're dying out here!

- parisc rp5470
- hpux 11.0
- oracle 8i
- iplanet webserver
- java 1.3

We're having a severe performance problem when
many webusers place a significant load on our system. Glance tells me cpu/disk/memory are not exhuasted, and database performance outside of the web interface is very good.

We suspect java garbage collection is inefficient and possibly our own code is keeping the java heap unusually full,

BUT

working on the theory that it might be kernel paramater limitations, would UX ever tell me anything if maxdsiz or maxssiz were too low?

Would I see errors in syslog, etc? If the OS has no way of notifying me of exhausting a resource, what might the symptoms of a too low
maxdsiz/maxssiz be on a system running a java/web app talking to an oracle database...?

Thanks very much - this problem is killing us on a production machine!

[Howard Palmer]

maxssize too small etc would manifest itself as application errors. It will depend on how your particular application handles a "can't allocate shared memory" error. Perhaps your app has a logfile somewhere?

It's unlikely to appear in syslog unless your application explicitly writes to the syslogd, or a kernel/system component has the same issues (very unlikely).

[Howard Palmer · Accepted solution]

You can also use glance and choose the "t" option to view kernel tables, see if anything there is at or near capacity (eg file locks etc, causing processes to have to wait?)

[Muthukumar_5]

maxdsiz and maxssiz are related on maz data size and stack size for 32 bit process.

Another set for 64 bit is available there out.

You can get kernel module details as kmtune -q -l

And more what is your user limit of ulimit -d and ulimit -s for hard and soft limit saying.

ulimit -Hs ( hard stack size )
ulimit -Sd ( soft data size )


what is your virtual memory status of vmstat saying?

You are being 11.00 so if you want to tune kernel you have to rebuild them out too.

[Simon Hargrave]

Sorry the above answer was from me, I was logged in as the wrong user!!!

maxssize too small etc would manifest itself as application errors. It will depend on how your particular application handles a "can't allocate shared memory" error. Perhaps your app has a logfile somewhere?

It's unlikely to appear in syslog unless your application explicitly writes to the syslogd, or a kernel/system component has the same issues (very unlikely).

[John Palmer]

When you say that cpu/disk/memory are not exhausted, what IS the bottleneck on your system.

Is it that you have multiple CPU's but one particular process is maxing out one of them?

Regards,
John

[Simon Hargrave]

Sorry the above answer was from me, I was logged in as the wrong user!!

You can also use glance and choose the "t" option to view kernel tables, see if anything there is at or near capacity (eg file locks etc, causing processes to have to wait?)

[Tim Killinger]

John Palmer,

Yes, we see very high CPU utilization on the single ns-http process of iPlanet. Other than that all we see (so far) is a steady degradation of response time to the web users.

[John Palmer]

Hi Tim,

I don't know anything about iPlanet I'm afraid but things that I'd be interested in are:-

1. Is there any spare CPU on the server?
2. Is iPlanet multi-threaded or multi- processes?

It sounds to me as though your web server is max'd out, nothing to do with kernel parameters. I'd be looking to see if the throughput/performance of iPlanet can be improved by any means e.g. allocating it more memory, allowing more threads etc.

Good luck,
John

[Steve Lewis]

Sounds like a typical case of the programmers trying to blame the system config for their poor code.

1. Use the utilities pointed to by the posts above to prove that the parameters are not maxed out.
2. sar -v 3 1 ; sar -c 3 1 ; vmstat -S may also help. If you have seen any swapped out pages then you will need more memory.
3. Check that you have the latest Java patches. I would even test the system under Java 1.4 and use that instead if it is OK.
4. Check you network interface speed/duplex settings.
5. Be aware that Java is memory hungry and runs more slowly than compiled C code regardless of the platform. I would not recommend anything less than two 1Ghz cpus and 8Gb of memory for your applications.