Priority Calculation for timeshare processes

[Prashanth C]

On our system, all applications are timeshare processes. We found a CPU bottleneck at one point of time. One process hogs a lot of CPU, and others starve for a few seconds. We tried using nice values for different applications. This improved the situation, but has not solved the problem.
We are trying to understand how the priorities are internally calculated by HP (we are using 10.20).
Is there some document available on how priorities are calculated depending on the CPU usage?

Thanks,
Prashanth

[keith persons]

Prashanth,

Look in /usr/share/doc for proc_mgt.txt. The description of process scheduling is outlined there.

Regards,

Keith

[Prashanth C]

The white paper is quite useful. It gives us the general mechanism of process scheduling. But, this question was specific to the algorithm or heuristic that HP-UX uses to calculate the timeshare priorities.
Other than nice, is there any other mechanism to control the priorities of timeshare processes? Either statically or at runtime?

[James Murtagh · Accepted solution]

Hi Prashanth,

I'm not going to repeat everything the Process Management White Paper says, but what I will mention here is better understood looking at the Timeline section under the Thread Scheduling chapter.

The formula used by calcusrpri() is:

pri = (kt_cpu/4) + PUSER + 2*(p_nice - NZERO)

A little description of the terms:

PUSER = base priority of user processes = 178
NZERO = 20
pri = process priority
p_nice = nice value of process
kt_cpu = cpu usage estimator, incremented by one each clock tick. Looking at the diagram I mentioned shows setpri() is run every 40ms, hence the formula value.

You can find a lot of this in /usr/include/sys/param.h

You won't find too much more without doing an Internals course I'm afraid. It may also be worth getting a Unix Internals book, a good one I always mention is:

Unix Internals: The New Frontier by Uresh Vahalia.

You would see from this the formula is very similar to the SVR3 implemtation.

Regards,

James.

[Jos Francois]

We have processes running at priority 152, eating cpu as hell. As far as I can notice, the priority of those processes doesn't change, even with a nice value of 39. When I look into /usr/include/sys/param.h, I find something that (maybe) could influence the base priority of a process ie. PRIBETA (6+PTIMESHARE) PUSER (50+PTIMESHARE) and so on. This latter gives a (base) priority PUSER of 178. Are those definitions indeed base priorities? And if so, how can we assign a base priority to a process?

[James Murtagh]

Hi Jos,

It might be worth asking this is a new thread but I'll try and answer anyway...

Timeshare processes have the external range 128-255. By external I mean external to the kernel and what you see from tools like ps. The base priority is TIMESHARE which is equal to 128. Values from 128 to 177 are termed system level priorities. Values from 178 (PUSER) to 255 are termed user level priorities. There is also a key priority at 153 - processes that have a priority equal to or below this are not signalable. A good example here is the shell - if you check the priority of your shell using ps, you will no doubt see it at 178 as it is in user land. If you check it from another terminal when it is idle it will be at 154, the highest priority it can be that will allow it to be signalled.

Anyway, I digress....looking at the timeline in the white paper the setpri() routine is called just before calcusrpri(). The setpri routine only adjusts the priority if this is greater than PUSER, or 178. So it appears that system level priorities do not change. Also, user level priorities can be made to be of type SCHED_NOAGE (see /usr/include/sys/sched.h) which means they have a fixed user level priority. To find out the scheduling policy I think you need to walk the kernel. I'm on 11i, have taken pid 1571 at random (its actually perflbd):

q4> load proc_t from proc_list next p_factp max nproc
q4> keep p_pid == 1571
q4> print -x p_schedpolicy
p_schedpolicy 0x2

So 0x2 is timeshare for the sched.h header file.

These priority definitions will be set in the source code I believe, so I don't believe you will be able to dynamically modify these, however a programmer may be better placed to answer this. A good starting reference may be the pthread man page. I suspect the problem using a lot of CPU will not be down to its given priority per-say, but some other event. If you let me know the name of this process it might be a known issue.

regards,

James.

[Jos Francois]

Thanks. I posted this as a new thread.