Help on the ENQUEUE parameters

[roose]

Folks,

Just recently encountered a problem on one of our applications, wherein we saw the error message

-RMS-F-EXENQLM, exceeded enqueue quota

Although the root cause has already been attributed to an application bug, several questions arised while we were doing root cause analysis related to this particular parameter. Can somebody share their ideas on the following questions?

1. Doing a help/message for this particular error suggests that you modify the UAF parameter ENQLM for the account that is encountering this error. However, researching further on this, aside from the UAF parameter ENQLM, you still have the dynamic SYSGEN parameters PQL_DENQLM and PQL_MENQLM. If I modify the UAF paramter, will it be superceded by any of these SYSGEN parameters?

2. Documentations shows that PQL_DENQLM is the default enqueue limit for those processes created by the $CREPRC service and DCL command RUN, whereas PQL_MENQLM is the defult limit for the number of locks that can be queued at one time.

- for DENQLM, this will only apply if you created or executed a run command to create a process without the /enqueue parameter with it, correct? If this is so, if the UAF parameter ENQLM for the account this process was run under has a higher value, will it supercede the DENQLM limit, or will it be limited to the DENQLM value?

- for MENQLM, somebody from HP told us that this is actually the minimum value that a process can use to set its ENQLM value, and is actually not a hard limit wherein if all processes on the system consume the MENQLM value, then consequences, which I don't know, will start to come in. Is this information correct?

3. What other parameters or resources, i.e. memory, will need to be changed when changing the UAF ENQLM parameter? How about PQL_DENQLM and PQL_MENQLM? I tried changing thes parameters and inserting them into MODPARAMS.DAT and run autogen but did not gave any hint which parameter was related to these.

Thanks in advance for your help!

[roose]

By the way, I forgot to ask, what MONITOR class should I use to monitor the overall ENQUEUE usage on the system?

Thanks again.

[Arch_Muthiah]

Roose,

You can use these comnds to monitor overall enq usage.
$ mon dlock
$ mon lock

Archunan

[roose]

Thanks for the reply Archaunan.

I tried doing a monit lock and I see that the value for total locks is a bit higher than the total resources:

Total Locks 6457.00 6456.00 6451.00 6463.00
Total Resources 5332.00 5331.57 5330.00 5333.00

Would you know what this means or how to interpret these data? I am over-using my system wide (PQL_MENQLM) limit?

[John Gillings]

Roose,

PQL_DENQLM is only relevant for processes that are created without specifying an explicit ENQLM. Any process created using a UAF entry specifies all quotas, so none of the PQL_D* parameters have any affect.

PQL_MENQLM affects all processes. If the effective ENQLM for the process (from explicit $CREPRC parameter, UAF or PQL_DENQLM) is less tham PQL_MENQLM then the process will be raised to the minimum value. Same for all PQL_M* parameters.

In general, it's best to avoid changing any of the PQL_* parameters, because doing so makes your processes dependent on a SYSGEN setting. This is not immediately apparent, and will probably cause trouble if/when you move the process to another system.

It's much better to make all necessary quota settings in the UAF, or in the $CREPRC call (or equivalent RUN/DETACHED command) used to create the process. This means they're documented somewhere, and will typically affect only the relevant processes, not everything on the system.

The only PQL parameter that AUTOGEN cares about or changes are PQL_*WS* (ie: the working set values). It's best to leave them be.

Obviously you should give a process enough ENQLM to do its job. But, if you see EXENQLM conditions, you should first check that the process isn't "leaking". Track the lock using using $GETJPI, item code "ENQCNT". One strategy is to raise ENQLM very high, run the process and monitor usage. See where it plateaus to determine a reasonable quota. Just make sure ENQCNT doesn't just keep decreasing. If it does, look for a leak.

If you have many processes using many locks, you may need to adjust LOCKIDTBL, but it's probably better to let AUTOGEN do that for you, based on the feedback statistics. The lock table will expand on its own, so unless it's very seriously under allocated, there shouldn't be any really bad consequences.

If you want detailed analysis of your locking parameters, please log a case with the customer support centre.

[Arch_Muthiah]

Roose,

The $PQLDEF macro defines these two parameter this way....

"Minimum" values are named PQL_Mxxxxx.
where xxxxx are the quota name.

"Default" values are named PQL_Dxxxxx. and xxxxx are the quota name.

So these two parameters values are set in SYSGEN as minimum and default value as "default", and "minimum". you can see
$MCR SYSGEN
SYSGEN> show pql_denqlm

If you don't speficy defalut value, the minimum value be assigned.
And there is "current" value in sysgen, that is the current value out of the value specified in the particular user's UAF record.
So ENQLM is the limit (maximum allowed) for that user, his processes and his sub-processes. But not for detached process.

Archunan

[Arch_Muthiah]

Roose,

could you please send your ENQLM limit set in UAF and PQL_DENQLM sysgen param?

If you have VMS version 7.1 and above, then you can set ENQLM limit to a value of 32767 in the SYSUAF.DAT file, OpenVMS treats this value as unlimited allows your application to use up to 16,776,959 locks, the architectural maximum allowed by the OpenVMS lock manager. Sub-resources and sub-locks greater than 65535 allowed.

Archunan

[Arch_Muthiah]

What is your env? details.

If you have vms version prior to 7.1, then ENQLM of 32767 used to be the maximum you could set in authorize. But you could double this by using run/enqueue=65635.

Archunan

[roose]

Archunan,

ENQLM for the account in question has now been set to 3500. Sysgen parameters PQL_DENQLM and PQL_MENQLM are 300 and 8192 respectively.

Understand as well that the process that was having problem is running under this account but is created through the DCL run command with the parameter /enqueue_limit set to 7000 previously, and now set it to 15000.

[roose]

We're running it on an Alpha ES80, 4GB memory and OpenVMS v7.3-2

[Arch_Muthiah]

Roose,

If you have VMS 7.3-2 then no need to restrict enqueue_limit to 7000, also set your ENQLM limit to 32767. Becauase when you give this(32767) limit, VMS will give unlimited locks to the processes under the user. Detached processes/its-subprocess if any, the locks won't be deducted from this limit. That is totally difft count.

If you still get the same error, then you will have to look into how many shared files your applications kept open and multibuffer count. and how many detached processes your application creates.

wish you solve your problem, see you tomorrow

Archunan

[Volker Halle · Accepted solution]

Roose,

you can check the current ENQ quota and limit values of your running process at any time with:

$ ANAL/SYS
SDA> SET PROC/IND=
SDA> SHOW PROCESS
SDA> READ SYSDEF
SDA> FORMAT JIB
...
FFFFFFFF.8302A558 JIB$L_ENQCNT 000007CF
FFFFFFFF.8302A55C JIB$L_ENQLM 000007D0
...

ENQLM is the limit (max. no. of locks allowed = quota) and ENQCNT is the current remaining quota (can also be seen with $ SHOW PROC/QUOTA/ID=

You could also see all the locks the process currently owns with:

SDA> SHOW PROC/LOCK/BRIEF

Volker.

[roose]

Guys, thanks for your replies! This has some brought great amount of insight for me regarding this particular parameters.

Some last questions:
1. What is the impact of changing these parameters, UAF and SYSGEN, on the node's resources? Meaning, if I raise the UAF ENQLM and/or the SYSGEN ENQLM parameters, how much will it impact my memory? Or any other parameters that we will need to watch out for?

2. As Archunan has mentioned, with our current OS, I can actually raise the UAF's ENQLM to 32767. If I do this, however, and a particular process did consumed this because of a problem somewhere, how will this affect the other processes, on different usernames, running on the same node? This is actually related to my question #1.

3. Lastly, Archunan mentioned that using monitor lock and monitor dlock will allow me to monitor overall system usage for enqueue. Which particular lines on the display will tell this, the total locks, the total resources, or another line? Also, will the values diplayed will have a 1-to-1 correspondence with the units (locks) used by all the processes during the time of the monitor display? Meaning, this monitor value are the summation of all the lock usage on the node during that particular time?

Again, thanks for your help on this matter.

[John Gillings]

Roose,

> What is the impact of changing these
>parameters, UAF and SYSGEN, on the node's
>resources?

On a system with 4GB of physical memory, your lock usage will trivial. Remember these mechanisms were designed when even a few MB of memory was very expensive.

> how will this affect the other processes,
>on different usernames, running on the
>same node?

You only need to unlimit the process if it really needs more than 32K locks. That's a lot of locks! Try setting the limit to (say) 32000 (ie: high, but not unlimited). If you still run into the limit, check for leaks.

> the total locks, the total resources,
>or another line?

Resources represent the things protected by locks. Locks are queued against the resource. So, if there's more than one process interested in a resource, you'll have more than one lock against it. So, typically the number of locks will be greater than the number of resources.

The size of the resource table is SYSGEN parameter RESHASHTBL. Each entry costs 4 bytes. You want it to be at least 4 times the peak number of resources. It's a cheap parameter, so don't worry too much about overallocation. Consider, RESHASHTBL=262144 will cost 1MB of memory. In your memory environment, that's nothing to be concerned about. Similarly LOCKIDTBL controls the lock ID table. Again each entry is only 4 bytes. The lock table will expand if required.

Don't stress too much about tuning these values! It would be quite hard to generate a workload (even a buggy one) that would cause much trouble with the resources you have.

Watch MON LOCK and track the MAX value. Also gather AUTOGEN feedback and look at "peak locks":

$ @SYS$UPDATE:AUTOGEN SAVPARAMS SAVPARAMS
$ SEARCH SYS$SYSTEM:AGEN$FEEDBACK.DAT LOCK

[Arch_Muthiah]

Roose,

< What is the impact of changing these
< parameters

There is no limit for ENQLM and the corresponding SYSGEN para can also be increased, but better to keep the default always. I think ENQLM will go upto 1 million.

Unless otherwise a application designed to be used by so many users, and their processes - subprocesses from very BIG cluster, sharing the disks, sharing the files, too many indexed files, and poor untuned SQL queries, long COMMIT intervals, etc, etc..., there won't be problem.

If we take care of lock contentions at disk, root index bucket, data bucket, files, and record levels, designing file access types, choosing the best compiler for reading/writing a BIG matrix (of size like 10Gb*10Gb) ROW-wise or Column-wise, etc, etc...., we can avoid lock realted problems.

This is the best way to avoid any issues related with any param including ENQLM, if the application desinged well, then it won't affect any other mem related sources, but there are few other param which will go along with this param, those are BIOLM and DIOLM. This two params will help reduce the lock contention problem while we do file 'WRITING'; and FILLM.

Archunan

[Arch_Muthiah]

Roose,

< how will this affect the other processes, < on different usernames, running on the
< same node?

Each user will have their own ENQLM limit.


Archunan

[Arch_Muthiah]

Roose,

< Which particular lines on the display
< will tell this, the total locks, the
< total resources

monitoring lock is node based. "total locks" shows the total locks on the node. If you want to see lock activities on hte other node, we have run this cmd in all the nodes.

If you want see lock "tree" activity across the cluster, then there is procedure (LOCK_ACTV.COM) which can be used to collect the lock activity per node and we have to summarise then to get the cluster wide lock activities.

try the following SDA comnds may give some good idea about lock identification, resource name locked, associated device, and file name, lock rate across cluster and across node...

sda> sho resource
with
/contention -- shows resources which are under contention

sda> sho lock
with
/waiting qualifier --- shows the lock which are blocked by other locks

sda> lck show active --- shows lock activity


Btw, may I know what happened to your "exceeded enqueue quota" error?


Archunan