topic Re: BYTLM / BUFIO byte count/limit strange behaviour in Operating System - OpenVMS

BYTLM / BUFIO byte count/limit strange behaviour

VMSRox — Fri, 28 Jul 2006 14:05:26 GMT

Hello OpenVMS friends,

I am seeing some behaviour that seems strange to me when monitoring process quotas for a detached process started using the following quota values:

/extent=14336-
/maximum_work_set=8192-
/ast_limit=256-
/buffer=450000-
/io_buffered=128-
/io_direct=128-
/subprocess=20-
/priority=4-
/enqueue_limit=10000-
/file_limit=512-
/page_file=350000

This process is a parent process that launches sub-processes when it receives requests. If I examine the process in SDA, or monitor it using the very handy "SHOW_QUOTA.COM" command procedure immediately after starting the process, it properly shows the BYTLM quota as 450,000 as I would expect.

If I check again later in the day, after many subprocesses creations / deletions, the BYTLM quota does not display 450,000, but rather shows some seemingly arbitrary number that is lower than 450,000. For example, I just checked, and the value shown is 128592 (see attachment). The process does seem to be running properly still, and all other quota values display properly. If I examine bytlm using the F$GETJPI lexical function, it also returns the same 128592.

Any ideas why the quota value is not being displayed properly? This doesn't seem to be impacting functionality, but it is a bit unnerving.

Thank you in advance for any help! Best wishes!

Re: BYTLM / BUFIO byte count/limit strange behaviour

Ian Miller. — Fri, 28 Jul 2006 14:11:44 GMT

There are certain operations which will decrement the quota e.g creation of a temporary mailbox. In this case when the mailbox is deleted the quota will be returned.

Re: BYTLM / BUFIO byte count/limit strange behaviour

VMSRox — Fri, 28 Jul 2006 14:42:30 GMT

Hi Ian,

Thank you for the quick reply. I don't understand why the Quota value would be decreased. Shouldn't it stay static? I understand that the "Used" value will fluctuate as sub-processes are created / deleted. It seems that the other quota values (PGFLQUOTA, TQLM, ENQLM, BIOLM, etc.) are static with fluctuating Used values. Why is BYTLM any different?

Thanks again,

Steve

Re: BYTLM / BUFIO byte count/limit strange behaviour

Jan van den Ende — Fri, 28 Jul 2006 15:05:37 GMT

VMSRox,

I can not check to make 100% sure at this moment, but ISTR that BYTLM is one of the "pooled" quota.
9volker and/or John will step in for sure if I am wrong!)
Every time a subprocess gets created, half of the CURRENT value goes into the child, half stays with the parent.
So, each child gets progressively less.
Some child dies (any reason), its quota gets returned (but the amount depends upon how much it got when created).
Many (at least, if random!) subprocess creations and deletions, that slowly progresses to a value of (more or less) initial-quota / number-of-processes per process.

hth

Proost.

Have one on me.

jpe

Re: BYTLM / BUFIO byte count/limit strange behaviour

Volker Halle — Sat, 29 Jul 2006 02:44:11 GMT

VMSrox,

BYTLM is a 'pooled' quota. This means that it covers the whole job tree (master process and all of it's subprocesses). The pooled quotas are kept in the JIB (Job Information Block) in memory (in nonpaged pool). All processes in the job tree share the SAME JIB.

$ ANAL/SYS
SDA> SET PROC/IND=
SDA> SHOW PROC
SDA> READ SYSDEF
SDA> FORMAT JIB
...
83397BA0 JIB$L_BYTCNT 0003B350
83397BA4 JIB$L_BYTLM 0003C3D0
...
83397C00 JIB$L_ORG_BYTLM 0003D090
...

For the field definitions of the JIB see:

$ SEARCH sys$library:lib.req jib$l_

Certain operations seem to also deduct from JIB$L_BYTLM (quota), not only from JIB$L_BYTCNT (remaining count).

Volker.

Re: BYTLM / BUFIO byte count/limit strange behaviour

Wim Van den Wyngaert — Sun, 30 Jul 2006 11:15:56 GMT

I can confirm that BYTLM quota is decreased by e.g. Sybase server.

Wim

Re: BYTLM / BUFIO byte count/limit strange behaviour

VMSRox — Sun, 30 Jul 2006 18:17:05 GMT

Hi,

Thank you for the replies. It still isn't clear to me when or why the BYTLM quota is decreased. If it is decreased, how can I predict what to set it to so that I don't run out of BYTLM - just set it to an arbitrarily high number?

It seems to be inconsistent in this particular application. I have seen 20 subprocesses start with no decrease in the BYTLM quota, and I have seen as few as 12 subprocesses start with a signficant (over 50%) decrease in BYTLM quota - please see attachment.

Is anyone aware of where this behavior is documented? I have examined the system managers manual as well as some of the system services documentation and cannot find mention of the BYTLM quota decreasing.

Thank you again for your help. As always it is greatly appreciated. Best wishes always!

Re: BYTLM / BUFIO byte count/limit strange behaviour

John Gillings — Sun, 30 Jul 2006 21:50:08 GMT

VMSRox,

>Thank you for the replies. It still isn't
>clear to me when or why the BYTLM quota is
>decreased.

BYTLM is "special" and, yes, it's inconsistent with other quotas, and confusing to the uninitiated. But, that's just the way it was implemented, so the best you can do is learn how it works and how to live with it.

For ALL other quotas the *LM value is static and the *CNT value decreases as the quota is consumed and increases as it's released.

BYTLM is "special" in that BYTLM value is NOT static. As Volker has pointed out, the original BYTLM is stored in JIB$L_ORG_BYTLM, which IS static (but there aren't any standard interfaces to read the value - the easiest is in SDA)

BYTLM controls how a process may allocate non-paged pool memory (NPAGEDYN). There are two kinds of usage of non-paged pool. The "normal" type, for example, a buffer for an I/O operation. They behave much like other quotas. They are decremented from BYTCNT on allocation, and returned when the buffer is released. Typically the lifetime of the allocation is short.

Other usage is for "long term" objects, like mailboxes, network connections and other pseudo devices. When one of these objects is created, the allocation is decremented from *BOTH* BYTCNT and BYTLM. (and both are incrememted on deallocation).

So, the difference between JIB$L_ORG_BYTLM and BYTLM can be thought of as "permanent" or "long term" allocations from NPAGEDYN. The difference between BYTLM and BYTCNT are "short term" allocations, expected to be returned soon.

This can be useful in predicting resource availability across all the processes in the job tree sharing the pooled resource. BYTCNT is what's free right now. You can "expect" up to BYTLM bytes to be available for allocation in the near term.

> If it is decreased, how can I predict
>what to set it to so that I don't run out
>of BYTLM - just set it to an arbitrarily
>high number?

When allocating initial BYTLM you need to take into account long term and short term usages, across all processes in the job tree. In your case, it looks like the process is creating a few hundred K of mailboxes or network devices. The remainder is available for I/O buffers and other ephemeral allocations.

This behaviour is documented in the Internals and Data Structures Manual.

Re: BYTLM / BUFIO byte count/limit strange behaviour

John Gillings — Sun, 30 Jul 2006 21:57:01 GMT

Re Jan,

>Every time a subprocess gets created, half
>of the CURRENT value goes into the child,
>half stays with the parent.

Correction, (as requested... ;-)

Yes, BYTLM is a POOLED quota. As explained by Volker, that means the values are stored in the JIB, which is physically shared between all processes in the job tree. BYTLM is NOT subdivided between processes. ORG_BYTLM, BYTLM and BYTCNT are the same storage cell.

What Jan describes is called a DEDUCTIBLE quota. It's a silly concept, and there's (thankfully!) only one quota that uses that model - CPU Limit. Whoever dreamed up the concept clearly doesn't understand exponential growth or decay. Hopefully most people have infinite CPU limits, so they're not affected by the fragmentation of CPU as they go down levels in the process tree.

Re: BYTLM / BUFIO byte count/limit strange behaviour

Volker Halle — Mon, 31 Jul 2006 01:04:25 GMT

Steve,

John wrote:

This behaviour is documented in the Internals and Data Structures Manual

Chapter 25.4 talks about Mailbox creation and deletion. It mentions the IOC$CHKMBXQUOTA routine, which invokes the special EXE$DEBIT_BYTCNT_BYTLM_NW routine, which deducts both JIB$L_BYTCNT and JIB$L_BYTLM.

If you have access to the source listings, you could look for callers of routines, whose names start with EXE$DEBIT_BYTCNT_BYTLM, to definitely find out, which functions inside OpenVMS deduct both BYTCNT and BYTLM.

Volker.

Re: BYTLM / BUFIO byte count/limit strange behaviour

Wim Van den Wyngaert — Mon, 31 Jul 2006 01:50:18 GMT

And what uses BYTLM : mainly mailboxes.
So, if you have programs using lots of mailboxes (or network connections), do a stress test to see if the maximum quota is reached before the maximum number of mailboxes (process will go in RW).

I wonder how many tried this test with their Sybase server (the documentation is far from correct).

Also : window control blocks (mount/window=).
But I never tried that.

Wim

Re: BYTLM / BUFIO byte count/limit strange behaviour

Ian Miller. — Mon, 31 Jul 2006 03:49:46 GMT

I also see
JIB$L_PBYTCNT and JIB$L_PBYTLIM which appears to be 0 so I guess they are historical.

Re: BYTLM / BUFIO byte count/limit strange behaviour

Volker Halle — Mon, 31 Jul 2006 03:57:04 GMT

Ian,

JIB$L_PBYTCNT and JIB$L_PBYTLIM are described as Paged pool byte count / limit and apparently 'not implemented'.

Volker.

Re: BYTLM / BUFIO byte count/limit strange behaviour

VMSRox — Mon, 31 Jul 2006 13:11:16 GMT

Thank you all for the educational replies - especially John. That really cleared it up for me. I guess I will remove myself from the "unitiated" group for this one issue.

Best wishes!

Re: BYTLM / BUFIO byte count/limit strange behaviour

VMSRox — Mon, 31 Jul 2006 13:12:50 GMT

Uninitiated - maybe I'll put myself back in the group after than mis-spelling! Jeesh.