High load Average

[Coolmar]

Hi folks,

I have a couple of hpux 11.11 systems that are showing a very high load average (uptime). They are both in the 14-16 range, one of them got as high as 30. The thing is that there are no processes at 90-100% and SAR is showing the system to be pretty-much idle. The thing is that the system is running fine. In the past, whenever a system has a load average above 5, we would notice right away because the system would be so slow. Any ideas?

[Eric Antunes]

Hi Sally,

Can you post those 2 files:

#vmstat -dnS 6 100 > vmstat.log
#sar -bv 6 100 > sar.log

Best Regards,

Eric Antunes

[Olivier Masse · Accepted solution]

I've been seeing the same behaviour on my systems for the last two years or so. I even opened a call at HP about this but I didn't get any definite and satisfactory explanation.

In my opinion, the load average is no longer a good performance measure, at least when taken from the traditional tools (top, uptime, etc). Glance is more realistic when giving out performance metrics.

In the older years, having a load over 10 meant we were in trouble. It's no longer the case. I ran some tests with a special CPU hogging program taken from the HP Performance and Monitoring course (waste.c) and by tweaking it, I've been able to artificially inflate the load average to over 100 - and the system was still responding correctly.

The waste.c description in the source is that it "slips all of its processing in between clock ticks".

It seems that most modern processes, especially java-based, have lots of threads. I *THINK* that these threads might get executed within a very short period like the waste.c program does, and this might explain why we get high load averages.

In my case, I changed our monitoring scripts to report problems if the load average goes beyond 30. It's much higher than before, but it works well.

Good luck

[Coolmar]

Thanks Olivier, that makes sense as I have java on both those systems. I have a call open with HP and have to run some perfomance monitors, but it will just tell them the same as I see and have explained in the initial post.

Sally

[D Block 2]

also, keep an eye on the memory metric.. java tends to be resident memory based. might run glance and ps -elx to spot heavy memory consumers. if your free-memory becomes an issue, then consult the jvm experts to tune the starting parameters of java, such as heap-size, etc.. also, hit the http://www.hp.com/go/java site for know patches for performance.