topic Re: Acceptable values for CPU run queue length in Operating System - Tru64 Unix

Acceptable values for CPU run queue length

George Denyer_1 — Fri, 06 Feb 2009 15:42:33 GMT

I'm trying to find out what the acceptable values for the CPU run queue length are. I havent been able to find any HP documentation about it, and only a few articles in web but they dont agree, some say 1 per processor, some say 3 or 4 per processor.

So, can you guys please share your toughts on what are the maximum values that you consider safe, and what values would make you start worring, given that CPU utilization is high (close to 100%)?

Re: Acceptable values for CPU run queue length

Victor Semaska_3 — Fri, 06 Feb 2009 17:34:41 GMT

From my experience Iâ ve found that a run queue of up to 4 per processor has been acceptable. Once above that I seem to get complaints about poor performance when CPU is the bottleneck. It is highly subjective though as to what is considered acceptable performance. A lot depends on what your Alphas are used for. In my case our Alphas are primarily Oracle database servers.

Itâ s best to run some performance gathering utility like â collectâ and then check the data when poor performance is reported. That way you can determine what your threshold is.

Re: Acceptable values for CPU run queue length

Martin Moore — Fri, 06 Feb 2009 18:39:48 GMT

In my experience, anything up to 2 per CPU is no problem, up to 4 means keep an eye on it, and consistently over 4 indicates a CPU bottleneck. Of course, you might get an occasional spike even if there's no overall problem.

Martin

Re: Acceptable values for CPU run queue length

George Denyer_1 — Sat, 07 Feb 2009 00:06:55 GMT

Thank you Victor and Martin for your opinions.

Re: Acceptable values for CPU run queue length

Hein van den Heuvel — Sat, 07 Feb 2009 00:38:56 GMT

'it depends'.

You have to try interpret to run queue in the context of idle time and some application knowledge. A high run queue is never a great sign, but it might not me as bad as it seems.
It can literally be a matter of bad timing.
If lots of processes have a timer set to poll a status (Oracle's control-c by client for example), then they may all become runnable at the same time giving the impression of a big backlog, but no actual user might be waiting and they all turn around and set a next timer in a hurry leaving plenty of idle time and plenty of responsiveness for end users.
The Oracle redo-log group commit is an other one a little like that. Very simplistically... let's say the system is idle and user A commits a transaction into the redo log. Oracle starts a write. Now out of nowhere users B,C,D,...Z all commit their transaction. Oracle does not start an IO for each, but waits for the A commit to finish, and in the mean time appends the redo data for B,C,... Z to the redo buffer. Now when A is committed the slaver process for A is woken up and, the IO for B... Z is started. When that (large) IO finished, B...Z are all woken up at the same time! Suddenly you have a run Q of 25, and the average may look like 5. But most of the time you were waiting! Still, that example may suggest a problem, but where you conclude it is a CPU issue, the IO bandwidth may be more important than the CPU time in this case.

fwiw,
Hein.