Queues msteriously stopped

[vmsserbo]

Most of our batch queues were in a stopped state. When I enabled autostart they all started up? What may have caused the queues to stop? The queues were working fine yesterday?

[labadie_1 · Accepted solution]

You shoud have a message in operator.log.

$ search sys$manager:operator.log "-E-","-F-"

(or opc$logfile_name, if you have defined the logical)

[vmsserbo]

I did a search for sys$manager:operator.log "-E-","-F-"

No strings found?

[labadie_1]

the operator.log contains the time period of the problem ?

[Volker Halle]

Miles,

did the system crash and reboot without an $ ENABLE AUTOSTART/QUEUES command during startup ?

If queues get stopped due to some other error (like exceeding MAXPROCESSCNT), you would most likely have to explicitly start them again.

Volker.

[vmsserbo]

No The system did not crash. Very strange

[Volker Halle]

Miles,

did the queue-manager crash ? SHOW PROC/ACCOUNTING/ID= ? Does the Connect Time shown match the system uptime ?

Volker.

[vmsserbo]

Yes it does

[Volker Halle]

Miles,

is OPERATOR.LOG active ? Does TYPE/TAIL SYS$MANAGER:OPERATOR.LOG show entries from the current date ?

Volker.

[vmsserbo]

Yes it does!

[Volker Halle]

Miles,

$ SEARCH SYS$MANAGER:OPERATOR.LOG QMAN,QUEUE

any matches ?

Does $ ACC/PROC=BATCH/SINCE=YESTERDAY show any batch jobs with error status (may also need /FULL to get more information) ? With the accounting data, you could at least find out, WHEN or IF batch jobs were running yesterday or until which time they were still running.

Volker.

[vmsserbo]

SEARCH SYS$MANAGER:OPERATOR.LOG QMAN,QUEUE

It did find matches but not from yesterday. The dates were from the 6th and 13th of december.

NO batch queue errors

[EdgarZamora_1]

There is the possibility that someone stopped the queues. For example, here we have command procedures that could start and stop and modify all queues with a single execution of a procedure. A DISABLE AUTOSTART command could've been done. Could someone have possibly stopped them?

Just to avoid future problems you should check your startup procedures and make sure you have an ENABLE AUTOSTART command.

[vmsserbo]

That is a possiblity, I did check the startup and it is there.

Thanks everyone

[John Gillings]

Miles,

Although it won't help for answering retrospective questions now, if you have queues mysteriously changing state, don't forget you can audit them. Add an ACE to the queue so that any changes are logged to the audit journal:

$ SET SECURITY/CLASS=QUEUE queue-name -
/ACL=(AUDIT=SECURITY,ACCESS=MANAGE+CONTROL+SUCCESS)
$ SET AUDIT/AUDIT/ENABLE=ACL

This probably doesn't audit changes in the AUTOSTART state. A quick look suggests it may be possible to audit ENABLE and DISABLE commands, but it's a bit complicated. You'd have to make copies of QUEMAN.EXE, call them QUEMAN_ENABLE.EXE and QUEMAN_DISABLE.EXE and put audit or alarm ACEs on them for ACCESS=EXECUTE+SUCCESS. Now make a copy of SYS$UPDATE:ENABLE.CLD and modify the image for the ENABLE command to QUEMAN_ENABLE, and DISABLE to QUEMAN_DISABLE and apply it to the system wide DCLTABLES.

If you get an unexplained change in the future, you can then look at the audit journal to determine who might be responsible.