topic Re: Queues msteriously stopped in Operating System - OpenVMS

Queues msteriously stopped

vmsserbo — Thu, 21 Dec 2006 09:27:52 GMT

Most of our batch queues were in a stopped state. When I enabled autostart they all started up? What may have caused the queues to stop? The queues were working fine yesterday?

Re: Queues msteriously stopped

labadie_1 — Thu, 21 Dec 2006 10:40:54 GMT

You shoud have a message in operator.log.

$ search sys$manager:operator.log "-E-","-F-"

(or opc$logfile_name, if you have defined the logical)

Re: Queues msteriously stopped

vmsserbo — Thu, 21 Dec 2006 10:43:01 GMT

I did a search for sys$manager:operator.log "-E-","-F-"

No strings found?

Re: Queues msteriously stopped

labadie_1 — Thu, 21 Dec 2006 10:49:19 GMT

the operator.log contains the time period of the problem ?

Re: Queues msteriously stopped

Volker Halle — Thu, 21 Dec 2006 12:06:16 GMT

Miles,

did the system crash and reboot without an $ ENABLE AUTOSTART/QUEUES command during startup ?

If queues get stopped due to some other error (like exceeding MAXPROCESSCNT), you would most likely have to explicitly start them again.

Volker.

Re: Queues msteriously stopped

vmsserbo — Thu, 21 Dec 2006 12:08:25 GMT

No The system did not crash. Very strange

Re: Queues msteriously stopped

Volker Halle — Thu, 21 Dec 2006 12:11:44 GMT

Miles,

did the queue-manager crash ? SHOW PROC/ACCOUNTING/ID= ? Does the Connect Time shown match the system uptime ?

Volker.

Re: Queues msteriously stopped

vmsserbo — Thu, 21 Dec 2006 12:33:39 GMT

Yes it does

Re: Queues msteriously stopped

Volker Halle — Thu, 21 Dec 2006 12:43:35 GMT

Miles,

is OPERATOR.LOG active ? Does TYPE/TAIL SYS$MANAGER:OPERATOR.LOG show entries from the current date ?

Volker.

Re: Queues msteriously stopped

vmsserbo — Thu, 21 Dec 2006 12:44:58 GMT

Yes it does!

Re: Queues msteriously stopped

Volker Halle — Thu, 21 Dec 2006 12:56:34 GMT

Miles,

$ SEARCH SYS$MANAGER:OPERATOR.LOG QMAN,QUEUE

any matches ?

Does $ ACC/PROC=BATCH/SINCE=YESTERDAY show any batch jobs with error status (may also need /FULL to get more information) ? With the accounting data, you could at least find out, WHEN or IF batch jobs were running yesterday or until which time they were still running.

Volker.

Re: Queues msteriously stopped

vmsserbo — Thu, 21 Dec 2006 13:01:48 GMT

SEARCH SYS$MANAGER:OPERATOR.LOG QMAN,QUEUE

It did find matches but not from yesterday. The dates were from the 6th and 13th of december.

NO batch queue errors

Re: Queues msteriously stopped

EdgarZamora_1 — Thu, 21 Dec 2006 13:31:50 GMT

There is the possibility that someone stopped the queues. For example, here we have command procedures that could start and stop and modify all queues with a single execution of a procedure. A DISABLE AUTOSTART command could've been done. Could someone have possibly stopped them?

Just to avoid future problems you should check your startup procedures and make sure you have an ENABLE AUTOSTART command.

Re: Queues msteriously stopped

vmsserbo — Thu, 21 Dec 2006 14:01:09 GMT

That is a possiblity, I did check the startup and it is there.

Thanks everyone

Re: Queues msteriously stopped

John Gillings — Thu, 21 Dec 2006 16:32:04 GMT

Miles,

Although it won't help for answering retrospective questions now, if you have queues mysteriously changing state, don't forget you can audit them. Add an ACE to the queue so that any changes are logged to the audit journal:

$ SET SECURITY/CLASS=QUEUE queue-name -
/ACL=(AUDIT=SECURITY,ACCESS=MANAGE+CONTROL+SUCCESS)
$ SET AUDIT/AUDIT/ENABLE=ACL

This probably doesn't audit changes in the AUTOSTART state. A quick look suggests it may be possible to audit ENABLE and DISABLE commands, but it's a bit complicated. You'd have to make copies of QUEMAN.EXE, call them QUEMAN_ENABLE.EXE and QUEMAN_DISABLE.EXE and put audit or alarm ACEs on them for ACCESS=EXECUTE+SUCCESS. Now make a copy of SYS$UPDATE:ENABLE.CLD and modify the image for the ENABLE command to QUEMAN_ENABLE, and DISABLE to QUEMAN_DISABLE and apply it to the system wide DCLTABLES.

If you get an unexplained change in the future, you can then look at the audit journal to determine who might be responsible.