High Load Averages

[Timothy P. Jackson]

Hello everyone!

I know that this subject has been beat to death in past forums, so Please hear me out.

I have spent many hours on this both in research and reading the forums that have anything to do with Load Averages. Bill Hassel and Stephen Protter have shed a lot of light on what Load averages really are and how to read them. Here are my questions......

I am running HP-UX 11i Version 1 on a rp7420 with 16 Gigs of memory and 4 1Gig CPU's.

There are currently 2 users logged in and I have load averages of around 0.34. Do I have something configured incorrectly?

I down loaded the Performance Collection script from one of SEP's forum responses and ran it. Along with a ton of information I can only see one thing that concerns me.

The output from the "sar -b" command show that my "%rcache" is 100 and my "%wcache" jumps around a lot and does reach 100 quite often.

Is this good having it reach 100%

Any help would be greatly appreciated!

Tim

[Patrick Wallek · Accepted solution]

You don't really give many details here.

You say you have 2 users logged in and a load average of 0.34. What you don't say is what this server is doing and what the users are doing.

Is it a database server? Application server? NFS? Something else?

%rcache at 100 is pretty good. That means all reads are coming from buffer cache. BUT, the fact that it is 100 leads me to believe that you don't have a lot of data on this server and that there isn't much activity. The %wcache seems to me to bear this out. I would not expect %wcache to be near 100, but I could be wrong. How much buffer cache do you have? What are your min/max_dbc_pct kernel paramaters set to?

Without more information it is very hard to give any more advice. But with what we have I'd say this server is behaving pretty well.

[Bill Hassell]

> There are currently 2 users logged in and I have load averages of around 0.34. Do I have something configured incorrectly?

I don't understand the question. I can log onto your machine and create a load average of 4.0 in just a few seconds by running 4 copies of a do-nothing script:

while :
do
:
done

A load average os 0.34 means that during the measurement period (1,5,15 mins), some trivial tasks were run but not much else. Were you expecting 0.00?

> Along with a ton of information

The size of your machine means that a *LOT* of money was spent on it. Do yourself a favor and get the Glance package. You can spend a few weeks reading the book "HP-UX Tuning and Performance: Concepts, Tools and Methods" by Robert F. Sauers, et al and all those metrics can be more meaningful, but Glance is by far the best performance tool there is for HP-UX.

The sar measurements %rcache and %wcache indicate how many I/O requests were completed from RAM and requried no disk I/O. 100% is highly desirable but impossible on a busy machine. Since your machine is essentially idle, most everything needed by the programs that are running are found in RAM. More 'normal' numbers (when you have 500 users and 8000 programs running) will be 90% rcache and 65% wcache. With 16Gb of RAM, the first error to fix is the buffer cache size. Change these two kernel parameters:

dbc_max_pct=6
dbc_max_pct=2

(they are probably 50 and 5)

[Timothy P. Jackson]

Sorry for all the confusion and lack of information! I could post the results of the perf. script if that would help. So let me get right to the point.

The application that I am running on this machine has higher run times than the same package on a MUCH smaller machine with . Thus the reason for mentioning the machine size.

I can be running 110 users on the smaller machine and have run times way less then 2 users on the large machine.

I have already changed the db_min_pct and db_max_pct to 2 and 6 respectively. This was done before the application was installed.

So the two questions I had were why is this happening and if %rcache and %wcache reaching 100% is good. Bill answered the second question and I guess the answer to the first question is to install guide and see what it can tell me.

Thanks for all your help!

[Timothy P. Jackson]

I have more analysis to do in order to figure out why two machines running the same application have different load averages.

[Bill Hassell]

Just a note about comparing performance between machines. You need a lot of knowledge about what the applications are doing. If the apps are databases, they are *not* identical unless they share the same database *and* users type in the same commands. Having a lot more memory is meaningless until you know if the application makes use of the extra memory. There are just too many variables to make any recommendations. You need to install Glance on both machines and start comparing the same activity. My guess is that there are a lot of differences between the machines, especially for application configs and data.

[Timothy P. Jackson]

Thanks Bill!

The application files are not absolutely identical but the data base engine is and the processing *SHOULD* be fairly close.

The data base engine should be using the memory efficiently and on a per user basis.

Obviously, as you have pointed out, something has to be different. I just about have Glance Plus installed on each machine and then, hopefully, I can start to see some differences.

With my 4 CPU's at 100% idle and only two users logged in, there must be something running that I haven't found. Maybe a background process that one of the developers put out there and didn't tell me.

Again, I really appreciate you help!

Tim

[Bill Hassell]

Actually, you can just use top to find which processes are using resources. My guess is that they will things like top, vxfsd, swapper, diagmond, sendmail, ttisr, etc, none of which are part of Unidata. These are normal system processes and occasionally consume some CPU time. You might have some cronjobs or someone ran a very short report. Remember that the load average is a long term (1 minute minimum) metric and a single process might consume 100% CPU for a couple of seconds which will average out over the 1 minute interval. You will seldom see 0.00 load average. top (and Glance) will confirm this.

[Timothy P. Jackson]

Hi Bill,

This is the what has me bugged. I normally use Top, vmstat, iostat etc... but I just cannot seem to find any one or two processes that seem to be sucking up processor time.

I am hoping that maybe glance might show me something that I am not seeing otherwise. I haven't used "sar" in almost 15 years back in the days or Unisys on Unix System 5. This was some of my problem with reading the information that came out of the performance script

Any way, I now have Glance in place and I will start looking for differences in the two systems.

Thanks again for your help!

Tim