SCHED_NOAGE

[Karen Shen_1]

I have been recommended by my database vendor (Progress) to use an HP policy called SCHED_NOAGE to run all my OLTP database related processes. This is to prevent priority inversion supposedly.

Can anyone provide any more information on this policy? Any tradeoffs I should be aware of?

Thanks! I'm on HPUX 11.0 on a V2600 24way.

[Fragon]

Which version of Progress are you use?
Is it from MFG/PRO?

-Gerald-

[T G Manikandan]

/*

HP SCHED_NOAGE Scheduling Policy
On HP, most processes run under a time sharing scheduling policy. Time sharing can have detrimental effects on Oracle performance by descheduling an Oracle process during critical operations, for example holding a latch. HP has a modified scheduling policy, referred to as SCHED_NOAGE, that specifically addresses this issue. Unlike the normal time sharing policy, a process scheduled using SCHED_NOAGE does not increase or decrease in priority, nor is it preempted.

This feature is suited to online transaction processing (OLTP) environments because OLTP environments can cause competition for critical resources. In laboratory tests, Oracle9i performance increased by up to 10 percent in OLTP environments using the SCHED_NOAGE policy.

The SCHED_NOAGE policy creates little or no gains in decision support (DSS) environments because there is little resource competition in these environments. Because each application and server environment is different, you should test and verify whether your environment benefits from the SCHED_NOAGE policy.

Avoiding Priority Inversion


Priority inversion can be a significant performance problem for
applications that use mutexes or spinlocks. The use of non-degrading
timeshare priorities, as provided by HP-UX's SCHED_NOAGE policy,
practically eliminates priority inversions. Specific threaded
applications have shown a 25% to 100% improvement in performance when
using SCHED_NOAGE.


Priority inversion occurs when a low-priority thread blocks a high-
priority thread from running. This can happen frequently because of the
nature of timeshare priorities. Consider the following scenerio.
Assume thread A is running, and thread B is blocked. As a thread runs,
its priority degrades, so the longer thread A runs, the weaker its
priority gets. Thread B, meanwhile, is blocked, so its priority is
getting stronger. Now thread A obtains a mutex, and just then thread B
wakes up. Thread B (with a strong priority) involuntarily preempts the
weaker thread A. So thread A is switched out, but it still owns the
mutex. Thread B starts to run, but fails when it tries to get the
mutex, and now has to exercise the mutex system's sleep code.


PHKL_16239:
SCHED_TIMESHARE thread priorities degrade over time and with
cpu usage. SCHED_NOAGE provides a scheduling policy where
the priority does not degrade.

In order to utilize the SCHED_NOAGE feature, patch
PHKL_16236 must be installed along with PHKL_16239. The
order of installation is not important. If this feature
will not be used, PHKL_16236 is not necessary.



SCHED_NOAGE is available in all versions of HP-UX 11, but requires
patches in the earlier versions. In addition, the policy number of
SCHED_NOAGE is not in the header files even after the patches are
applied. For reference, SCHED_NOAGE is policy 8.


The SCHED_NOAGE policy follows the standard inheritance rules: that is,
the descendants of a SCHED_NOAGE process inherit that policy, unless it
is explicitly changed by the descendant. This can be done through the
sched_setparam() and sched_setscheduler() calls. See Appendix D for
source code that accepts a process ID as input, then changes the
scheduling policy of that process to SCHED_NOAGE. An easy way to start
up an application as SCHED_NOAGE is to set SCHED_NOAGE on the starting
shell, so its children will all inherit that policy.


*/

[Karen Shen_1]

My Progress version is 9.1C and it is not MFG/Pro.

Thanks.

[Jeff Schussele]

Hi Karen,

We were having priority degradation problems on systems running MQ Series 11.520.04 along with some home-grown C & Java apps on maxxed-out (32X32) V-class 11.0 systems.

We were experiencing CPU spiking problems & apparent transaction ceilings where we just couldn't push any more transactions through even though we were way below the forecast top-out.

HP advised to load 3 new pthread/mutex performance enhancement patches (PHKL_25998, PHKL_27510 & PHCO_26960)& implement the sched_noage policy when starting the heavy hitting processes. AT first usage this appeared to have quite a positive impact, but then we started to have problems with MQ queues hanging. We had to backout the patches & the solution now appears to be to throttle back the amount of application processes being spawned. The root-cause apppears to be that the system was spending way too much time housekeeping for 3000+ plus processes & their associated MQ queues & backing them off to 2000+ pushed the actual work transactions much higher.

So I'd say that yes that using sched_noage & the pthread/mutex patches can help in certain situations, but that you really need to investigate the root cause of priority degradation - i.e. is the system spending too much time in System & not enough in User & MOST importantly - WHY?

Rgds,
Jeff