High CPU Utilisation

[Hoey]

Hi
We had 2 CPUs on our L2000, the utilisation always top to 98%. Then we added one more CPU, we thought the utilisation will be down, but surprisingly, the utilisation is still 98 sometimes 99%. Could anyone please explain why the situation is that.

Thanks
Davis

[Patrick Chim]

Hi,

Have you checked that which process(es)used up most of the CPU resources before and after adding the CPU ?

How about the output of glance/top/sar ?

Do the memory and disk utilization also high at that time ?

Sometimes a self-developed application program can use up to 100% CPU resources as I have seen before.

Regards,
Patrick

[James R. Ferguson]

Hi:

CPU utilization of 100% is not necessarily bad nor unexpected in certain situations.

A processor bound task without other interrupts (of which I/O is one) will be a constant candiate for the CPU and utilization will therefore saturate.

In your case, I would hope that the run-queue depth would have fallen with the addition of the third CPU. One metric of this is the 'uptime' statistic.

I would use 'glance' to determine what process or processes are consuming excessive CPU time and where they are utilizing the processor.

Regards!

...JRF...

[Patrick Chim]

Hi again,

I agree to James that CPU utilisation up to 100% may not be bad.

You can try to vi a 100M file and replace a frequent string and at the same time you monitor the system using 'top -s1' you will see that the vi process has used up to 98% of CPU utilization.

If you break the vi process, the utilization will drop again !

Regards,
Patrick

[KCS_1]

hi,

I have been very similar the probem like above and sovlved my problem that applied the "diagnostic(mstm version up)" patch on system.

what is your mstm(diagnostic tool) version??

you can apply to recently patch for mstm on your system.

try this under site:
http://software.hp.com

[RAC_1]

I would first ask what you are running on system

then what about memory,disk,network utilization?

[Bill McNAMARA_1]

use top to find out which process is consuming the CPU.
There were problems with diagnostics processes in the past.. (as mentioned), but we'll only be able to verify if you can identify the process.

Are the 2 CPUs at ~100% ?

Later,
Bill

[Todd Larchuk · Accepted solution]

There is a classic traffic engineering dilemna that by measuring the carried load you cannot estimate the offered load. In this case the carried load is the CPU utilization with 2 CPUs. By measuring the 98% utilization you actually do not know how much more utilization you would get if you had more CPU. If you have a very good performance model you might make a fairly accurate guess, but performance models are the exception rather than the rule.

By the way, high CPU utilization is a good thing. You paid a lot of money for a powerful server, why would you want to use it less than 100%?

In the area of performance, there will always be a bottleneck. If it isn't CPU then it will be something else (memory, disk i/o, network i/o, etc) that runs out first. If nothing is at 100%, then your server is too big! As long as your response times and throughput are acceptable, don't worry that some resource is fully utilized.

[Sridhar Bhaskarla]

Hi Davis,

You would need to do little more research. Your baseline prior to upgrading the CPU was not much accurate. You would have taken the runqueue (sar -q) before the upgrade and compare it afterwards. Also, along the same lines note the other bottlenecks on the system like disk I/O, memory etc., Perfview/Glance are very useful but sar can also be helpful. If your sar -u 2 20 output contains unusual numbers for system and wio, then it is more likely that you are experiencing bottlenecks in other areas that are causing high CPU. If you find any of them, you will need to work on fixing them that will automatically fix the CPU utilization.

-Sri

[Hoey]

Thank you all for the great and helpful reponse. I ran sar -q but how can I interpret the output.
HP-UX u1gtgam B.11.00 U 9000/800 09/20/02

09:58:03 runq-sz %runocc swpq-sz %swpocc
09:58:05 1.3 50 0.0 0

Our runq-sz is 1.3, is it high or low? And, what scale is regarded as high or low? Also, what exactly is %runocc, swpq-sz %swpocc means in plain english?

Thanks

[keith persons]

Davis,

First, it looks like you may have run sar with only one iteration - if this is true I would suggest running it again with at least 2 iterations. The first line of output from sar, vmstat, iostat, nfsstat should be ignored - it's the cumulative average since the last reboot and the numbers are not helpful.

runq-sz is the average length of the run queue - on average, how many processes are waiting to get time on the cpu.

%runocc percentage of time the run queue was occupied by processes waiting to get time on the processor

swpq-sz on average, how many processes are in the swap queue that are runnable, normally previously swapped out processes that now have all required resources and are waiting to get swapped back in

%swpocc percentage of time the swap queue was occupied by runnable processes.

So what do these numbers mean? By themselves, usually not much. They're usually of most help when factored in with overall utilization curves.

The sar data is most helpful to set generally acceptable ranges of values for your system - the baseline. Then watch for changes, deviations or trends or steady increases.

Hope this helps your understanding some.

Keith

[Sanjeev Kohli_1]

Hi Davis

The CPU utilisation depends on other factors to.
you need to look into memory utilisation of the server.
you need to check whether your machine is swaping ,if there is lot of swap activity then there is good number of I/O on the server which will utilise the CPU as it has to read data from the disk to put it in the memory for processing.
You need to check the process which is occupying the CPU.

Have fun,

Regards,

Sanjeev