topic Re: Setting TQELM limits in Operating System - OpenVMS

Setting TQELM limits

Richard Leighton — Tue, 30 Nov 2004 03:55:14 GMT

Hi,

A number of application specific processes were seen to be in a state of MUTEX. Via the SDA the the resource wait state was WTTQE. Therefore the TQELM limit was increased for the process' owner.

Ask the Wizard stated:

"
TQELM is the Timer Queue Entry Limit.

The TQELM is the maximum number of timed events you may have outstanding. Most processes don't require many, so quotas as
low as 10 can be reasonable. (Third-party software can require far more of these, of course.)

The most important thing about any quota is to make sure a process has sufficient to perform its job. As long as it doesn't actually
attempt to exceed the limit, there is no problem. Attempting to exceed TQELM or other similar quotas will put the process into a
resource wait state, when the next scheduled TQE expires, the request will be granted, and the process will continue. So this and other similar quota-induced stall conditions are self-healing, unless the process has timer response constraints.
"

As this issue has reoccured ... please could someone advise:

-What exactly are Timer Queue Entries?

-Is there a way to calculate the maximum number of TQEs a processes may need? i.e. is it code specific ..

-Is there an adverse effect on system performance by increasing this value?

Thanks in advance
R

Re: Setting TQELM limits

Kris Clippeleyr — Tue, 30 Nov 2004 05:08:10 GMT

Richard,

TQEs are data structs maintained by VMS in a doubly linked list, describing time-dependent requests, ordered by expiration time of the request.
Processes can use the system services $SCHDWK and $SETIMR to request time-dependent services; and can use $CANWAK an $CANTIM to cancel these requests.

The actual number of TQEs a process needs is indeed very dependent of the image that runs in that process. How many, hard to say.

Increasing the value of TQELM for a certain process and may afect the non-paged pool, since TQE are allocated from NPP. However, a TQE entry isn't that big, and if your Alpha has enough memory, I don't think that increasing the TQELM (to a reasonable value) will cause a performance issue.

Greetz,

Kris

Re: Setting TQELM limits

Ian Miller. — Tue, 30 Nov 2004 05:59:08 GMT

A TQE is only 64 bytes so allowing a process to have lots is not really a problem. The limit is there to prevent one process from using up all the non-paged pool (parhaps becase of a bug).

Sometimes the TQE limit is related to other quotas but this is very application specific. Just pick a number big enough for the application to work.

Re: Setting TQELM limits

Richard Leighton — Tue, 30 Nov 2004 06:28:58 GMT

Thanks Kris and Iain for your help.

Re: Setting TQELM limits

Wim Van den Wyngaert — Tue, 30 Nov 2004 07:17:18 GMT

It is also funny how VMS reacts.

1) when asking memory exceeding the pagefilequota : error returned to program
2) when asking SETIMR exceeding tqelm : hang and mutex.

I'm not a system programmer but shouldn't hang/error be a parameter for the call ?

Wim

Re: Setting TQELM limits

Ian Miller. — Tue, 30 Nov 2004 07:36:03 GMT

You can choose to disable resource wait mode so you get an error by using the $SETWRM system service. See
http://h71000.www7.hp.com/doc/732FINAL/4527/4527pro_007.html#index_x_1019

You can also disable this when creating a detached process e.g. RUN/NORESOURCE_WAIT

Re: Setting TQELM limits

Wim Van den Wyngaert — Tue, 30 Nov 2004 08:07:13 GMT

Ian,

Thanks for the clarification.
But it's still not clear :

1)if I specify /noresource, will it never wait, whatever the program specified ?

2) if the default is /resource, why is it for certain resources such as memory still "give an error" instead of waiting until memory is available ?

Wim

Re: Setting TQELM limits

Ian Miller. — Tue, 30 Nov 2004 08:40:45 GMT

resource wait mode only affects the behavior of system services for certain resources.
# System dynamic memory
# UNIBUS adapter map registers
# Direct I/O limit (DIOLM) quota
# Buffered I/O limit (BIOLM) quota
# Buffered I/O byte count limit (BYTLM) quota

The system services return SS$_EXQUOTA instead of waiting if resource wait mode is disabled.

Re: Setting TQELM limits

John Gillings — Wed, 01 Dec 2004 06:29:32 GMT

Richard,

Regarding Wim's query:

"1) when asking memory exceeding the pagefilequota : error returned to program
2) when asking SETIMR exceeding tqelm : hang and mutex."

There is a simple reason for this. PGFLQUOTA is a PROCESS quota. Therefore, if an allocation attempt fails, the same allocation will continue to fail forever.

TQELM and BYTLM are JOB quotas, pooled between all processes in the same job tree. So, if an allocation fails now, it may succeed later, if another process in the job tree returns some quota. (that VMS goes mutex even if there is only one job in the tree is questionable, but that's just how it works! at least the behaviour is consistent)

>What exactly are Timer Queue Entries?

You can request the operating system notify you at a specific time in the future, or after some time interval. The result may be setting an event flag, issuing a wakeup or firing an AST. The Timer Queue is a list of future notifications to be made. A JOB TREE has a limit as to how many notifications can be pending at any given time.

>Is there a way to calculate the maximum number of TQEs a processes may need? i.e. is it code specific ..

Yes it's code specific. You need to know how many future notifications are likely to be needed, and how long they will remain on the queue. Typical uses for timer entries are timeouts for I/Os and periodic events. I/Os tend to be short lived. Periodic events can be longer term. As timers expire they can be reused.

>-Is there an adverse effect on system performance by increasing this value?

On pre V7.3 systems, TQEs were kept in a doubly linked list, ordered in temporal sequence. If the queue got very long, operations to add entries at the correct place could take a long time (sequential scan required, at elevated IPL and holding the SCHED spinlock - not good!). A process with a high TQELM which added large numbers of entries could therefore adversly impact the whole system. A high TQELM which is not used does not affect the system.

As of V7.3, the TQE structure has changed, so insertions are far more efficient, even when there are large numbers of entries.

However, if you find you're exceeding TQELM even with high values, it's probably worth investigating exactly what the application is doing. Why is it issuing so many? Perhaps there's a time interval coded incorrectly (too long)? Are long term timers being requested and not cancelled when they should be?

Re: Setting TQELM limits

John Gillings — Wed, 01 Dec 2004 06:32:14 GMT

Oh, and to Ian's suggestion...

Please DO NOT disable resource wait mode, unless you have control over ALL system services your code will ever make. Behaviour of any RTL calls, and even some system services are unpredictable with resource wait mode disabled.

The purpose of $SETRWM is for inner mode and real time programming, it is NOT a magic wand for fixing application resource problems. See the System Services Reference Manual BEFORE attempting to use this service.

If you're not coding in MACRO-32 with IPL above 2 then you should NOT be calling $SETRWM!

Re: Setting TQELM limits

Wim Van den Wyngaert — Wed, 01 Dec 2004 06:50:28 GMT

John,

With mult-threaded programs it could be possible that memory comes available again.

The RW states are the main reason why we have to reboot nodes. It's a pain ...

Also : be carefull when changing quotas. Some products may refuse to start when they are not in balance (e.g. DSM).

Wim

Re: Setting TQELM limits

Ian Miller. — Wed, 01 Dec 2004 07:58:11 GMT

I agree with John about SYS$SETRWM - use it in your own application when you know what the affects are.
I mentioned it just to point out it is possibile to change VMS behaviour for resource waits not to recommend it.

Back to the original question - I would tend to increase the TQELM then watch the process with SHOW_QUOTA.COM/AMDS/Availability Manager to see what the behaviour is. The program may have a bug or it may use a lot of timers in its normal behavior.