Large "allocation" for SYS$QUEUE_MANAGER.QMAN$JOURNAL

[Art Wiens]

Before everyone jumps in and starts shouting DIAG 7!! DIAG 7!! my question is related to the "allocated" space for this file.

Alpha VMS v7.2-2 (4 Alphas using this queue manager).

Directory SYS$COMMON:[SYSEXE]

SYS$QUEUE_MANAGER.QMAN$JOURNAL;1
2/2010540 11-DEC-2006 17:09:30.49

This is after a DIAG 7, the consumed space wasn't much larger before I did it.

Can I make this file smaller "easily"?

Cheers,
Art

[Dale A. Marcy]

Actually, I thought that is what the diag 7 was supposed to correct. I think I have in the past (long time ago and fuzzy memory), stopped the queue manager, renamed the file and restarted the queue manager allowing it to create a default size file. The key is renaming the file, otherwise it creates another of the same size. It has been a long time since I have done this, so attempt at your own risk.

[John Gillings]

Art,

I think you just need to wait until the queue manager refreshes the file. I'm not sure if there's a JBC$COMMAND option to force that. Might be worth trying the DIAG 7 on the node running QUEUE_MANAGER?

Note the allocated space is probably worth less than US$1 so please don't lose any sleep over it!

[Art Wiens]

"worth less than US$1 so please don't lose any sleep over it"

Ahh, but those are them "full size" American dollars, our's are 20% smaller ;-)

I do have to worry a bit, the system disk is only on a 36GB mirror set. It's getting a bit tight on there with the 4 nodes contributing "debris".

I did perform the DIAG 7 on the node running the queue manager, and now that I look some 3+ hours later ... no diff:

Directory SYS$COMMON:[SYSEXE]

SYS$QUEUE_MANAGER.QMAN$JOURNAL;1
2/2010540 11-DEC-2006 17:09:30.49

Interesting the file hasn't been "modified" since I did the DIAG 7? I'm sure several hundred jobs have passed through in that time.

Been up for 211 22:54:47 ... I have no other reason to reboot. Would a reboot even fix this without intervention at that time? Renaming it etc.

It would be nice to have an extra 2M blocks on this disk "easily".

Cheers,
Art

[Steven Schweda]

COPY seems not to preserve the too-large
allocation. If you can stop the queue stuff
enough to stabilize the file, that might do
it for you. (Boot from a CD, on a bad day.)

You could do a BACKUP /IGNORE = INTERLOCK,
and then COPY the result, but that sounds
riskier.

[Art Wiens]

Would a JBC$COMMAND> diag 0 7 do anything for me? (instead of the often mentioned diag 7). I did this on a test system, it did create a new file (verified by file-id # change) however it's allocated size was the same as the original.

What does this file do? It hasn't been modified since the diag 7 on the 11th:

Created: 11-DEC-2006 17:09:30.49
Revised: 11-DEC-2006 17:09:30.49 (0)

"Lots" of batch and print jobs have executed since Monday.

Art

[Hein van den Heuvel]

Would a simple $SET FILE/TRUNCATE do the job?

That releases all clusters beyond the EOF.

Hein.

[Art Wiens]

It probably would if the file wasn't open:

$ set file/truncate SYS$COMMON:[SYSEXE]SYS$QUEUE_MANAGER.QMAN$JOURNAL;1
%SET-E-READERR, error reading SYS$COMMON:[SYSEXE]SYS$QUEUE_MANAGER.QMAN$JOURNAL;1
-SYSTEM-W-ACCONFLICT, file access conflict

Another interesting observation (I think) ... on my test system, the file is 2/108 blocks (used/allocated). If I shutdown the queue manager, delete the file and start the queue manager, the file gets recreated 2/108 blocks. The cluster size on that disk is 18. Where did it get 108 blocks from? ie. if my original diag 7 created a new file, why did it make it 2M blocks allocated?

Art

[John Gillings]

Art,

Oops, I forgot a VERY important point here... QMAN$JOURNAL doesn't play by normal RMS rules. The queue manager uses filespace BEYOND the RMS EOF mark, so the "normal" interpretation of used/allocated doesn't apply. Consider, do you really think you can store all the information about all your queue entries in just 2 blocks? Have you ever seen a journal file with an apparent used space with more than one digit?

One very valid reason that queue manager hasn't shrunk the file is it may actually be USING all that space!

So, if you've executed a DIAG 7, you should see some indication of the actual journal usage (or use DIAG 5 to show without performing a compression). It could be that you need to clear out some old entries before you can recover the allocated space. Could there be an accumulation of RETAINed entries?

Whatever you do, DON'T /TRUNCATE the file. That will make QUEUE_MANAGER very unhappy.

[Art Wiens]

"doesn't play by normal RMS rules"

This is why the file is 2 blocks big and showing as never being modified? Hard to believe that such a "violation" was ever allowed or needed for that matter.

"do you really think you can store all the information about all your queue entries in just 2 blocks"

No, but 2M blocks seems a bit much too. There are 336 queues in this environment (4 Alpha's sharing) and as of this moment, 46 queues have jobs in them in any manner of state (holding, executing, pending, stalled etc). There are 323 jobs in these 46 queues.

I tried the diag 5 and got:

JBC$COMMAND> diag 5
%JBC-F-SYSERROR, SNDJBC system service error at PC 000100A0
-NONAME-F-NOMSG, Message number 00000004

Tried the diag 7 with opcoms enabled:

JBC$COMMAND> diag 7
%JBC-I-DIAGNOSTIC,
Log for playback = 0
Save old Journal files = 0
Log all requests = 0
Dump on error = 0
Checkpoint: State = 1, In-memory blocks = 203005

$ %%%%%%%%%%% OPCOM 14-DEC-2006 22:22:45.44 %%%%%%%%%%%
Message from user SYSTEM on xxxxxx
%QMAN-W-LOWDISKSPACE, disk space is low on _$1$DGA61

$ %%%%%%%%%%% OPCOM 14-DEC-2006 22:22:45.44 %%%%%%%%%%%
Message from user SYSTEM on xxxxxx
-QMAN-I-FREEDISK, free up 4021080 blocks on disk _$1$DGA61

There are 5,970,370 blocks free on dga61:. I guess it's having trouble making the file smaller because the "lack" of free space?

"Whatever you do, DON'T /TRUNCATE the file. That will make QUEUE_MANAGER very unhappy."

Hein told me to! ;-) The set file /truncate I showed was on a test system - no worries. The info above is from the live environment.

I think I have some unused pagefiles in some roots. I can probably delete them and if so, I'ld like to try this again, but will it really want to make the file 4M blocks?! That will make the problem worse for me if that's it's final size.

Cheers,
Art

[John Gillings]

Art,

DIAG 5 should give the same output as DIAG 7, but without actually doing anything. I don't know why yours doesn't work:

JBC$COMMAND> diag 5
%JBC-I-DIAGNOSTIC,
Log for playback = 0
Save old Journal files = 0
Log all requests = 0
Self diagnostics: Memory status = 00000001, Index errors = 0
Dump on error = 0
Checkpoint: State = 0, In-memory blocks = 100

Your output says you have lots of journal stuff:

"In-memory blocks = 203005"

I can't remember how big each in-memory bolock is, but I know 200K is a lot!

>There are 336 queues in this environment (4 Alpha's sharing) and as of this moment, 46 queues have jobs in them in any manner of state (holding, executing, pending, stalled etc). There are 323 jobs in these 46 queues.

This sounds quite reasonable, and shouldn't take up mega blocks. Are you sure there aren't any retained jobs? Have you gone into extended range entry numbers at any time? What about very old jobs with outlier entry numbers? Again, I can't remember the details of internal journal file management, but it's possible a single outlier could force the retention of a lot of intervening space.

Maybe you could take a SHOW QUEUE/ALL/FULL, then delete entries that aren't going to start for a while and resubmit them. That might reduce the sparsness of entry numbers, making it possible to compact the internal space. Maybe also reset the entry number with a loop like this:

$ loop: SUBMIT/HOLD LOGIN.COM
$ DELETE/ENTRY='$ENTRY'
$ IF $ENTRY.GT.100 THEN GOTO loop

If you can manage to get down to 0 entries of any sort, even for a very short time, DIAG 7 should collapse the file to minimum size.

>Hard to believe that such a "violation" was ever allowed or needed for that matter.

Not really a "violation". It's a "live", non-shared, permanently open file. RMS only ever updates things like EOF and modification date when and if necessary. Since no processes other than QUEUE_MANAEGR have any need, or business to know, there's no need for those fields to ever be written. QUEUE_MANAGER may also access the file directly as a mapped section or $QIOs, rather than using RMS, in which case all bets are off about RMS structures.

Look at any privately owned file being written, what do you see? The allocation goes up, but the used size remains at 0. Similarly RDT = CDT no matter how old, or often the file is updated.