topic Re: Jobs stay in status starting in Operating System - OpenVMS

Jobs stay in status starting

Wim Van den Wyngaert — Thu, 19 Jan 2006 05:45:32 GMT

This morning I found a 7.3 cluster with all jobs in starting status.

I tried to restart the queue manager but the command hung.

I did a stop/id of the queue manager. It restarted but the jobs stayed in starting.

Nothing in system log.

I rebooted the queue manager node.

Problem solved.

Found that at the moment that the first job stayed in starting, a new accounting file was created.

Tried to simulate it by doing submits of jobs and set acc/new in a loop. After 3000 new accounting files no problems arrised.

Did someone already see something like this ?

Did someone see the creation of a new accounting file without doing "set acc/new" ?

Wim

Re: Jobs stay in status starting

John Abbott_2 — Thu, 19 Jan 2006 06:05:55 GMT

Not seen it.
How badly fragmented was your original acc file ? maybe this had something to do with it ? although I wouldn't have expected a new acc file to be automatically created after an extend error (one possibility).

Kind Regards
John.

Re: Jobs stay in status starting

Wim Van den Wyngaert — Thu, 19 Jan 2006 06:10:11 GMT

13770 fragments. I have other accounting files with 14094 and 14202 fragments.

Wim

Re: Jobs stay in status starting

Wim Van den Wyngaert — Thu, 19 Jan 2006 06:31:08 GMT

Uhm, skip that.

It's 346 fragments. But I have files with 390 fragments.

Wim

Re: Jobs stay in status starting

Wim Van den Wyngaert — Thu, 19 Jan 2006 06:42:21 GMT

Found something extra.

In the original accounting file accounting continued and the new one created yesterday only started to be used today at boot time.
But no batch jobs in it after the time the new file was created.

Wim

Re: Jobs stay in status starting

John Abbott_2 — Thu, 19 Jan 2006 07:05:30 GMT

If you copy away (save) the orig acc file, then try to write several records to the end of the orig file, does it work or do you get an error ? Just a thought.

PS. We create a new acc file every month as the acc file was the most fragmented file on our system, so I'm unlikely to see your problem.

Re: Jobs stay in status starting

Marc Van den Broeck — Thu, 19 Jan 2006 07:09:09 GMT

Wim,

is it possible that someone just created a new acc file with create or with an editor in stead off using set acc/new?

Rgds
Marc

Re: Jobs stay in status starting

Wim Van den Wyngaert — Thu, 19 Jan 2006 07:21:06 GMT

The blocks allocated to the original files are not yet all used. So yes, I can write to it. Our disks are defragmented weekly.

Marc : everything is possible. I tried to create the file by hand and it gave no side effects on accounting or qman.

Wim
(suffering from very slow internet)

Re: Jobs stay in status starting

Ian Miller. — Thu, 19 Jan 2006 08:55:27 GMT

could people login at that time (before you rebooted)?

Re: Jobs stay in status starting

Wim Van den Wyngaert — Thu, 19 Jan 2006 08:59:22 GMT

Yes Ian. All processes could continue running and could be created. Except batch ones. And the queue manager was running without problems too.

It gives the impression that the queue manager (or job_control ?) was waiting for the new accounting file to be operational. For ever.

Wim

Re: Jobs stay in status starting

Jan van den Ende — Thu, 19 Jan 2006 09:53:51 GMT

Wim,

All processes could continue running and could be created. Except batch ones.

How about PRINT jobs?
...trying to nail down or discard the queue manager...

Proost.

Have one on me.

jpe

Re: Jobs stay in status starting

Wim Van den Wyngaert — Thu, 19 Jan 2006 14:39:52 GMT

Jan

No print jobs in starting. But I will check Friday in accounting if any were executed during the night.

Wom

Re: Jobs stay in status starting

Wim Van den Wyngaert — Thu, 19 Jan 2006 14:40:01 GMT

Jan

No print jobs in starting. But I will check Friday in accounting if any were executed during the night.

Wim

Re: Jobs stay in status starting

Wim Van den Wyngaert — Fri, 20 Jan 2006 01:58:51 GMT

No print jobs. But no printjobs yesterday either. So, no conclusion.

Wim

Re: Jobs stay in status starting

Jean-François Piéronne — Fri, 20 Jan 2006 02:44:17 GMT

Jan,

may be no related, but

may I suggest to use SDA to take a look at the
QUEUE_MANAGER processes
for examples
sho process, show lock, show process/channel

What the size of SYS$QUEUE_MANAGER.QMAN$JOURNAL
If the file is very large you can try
$ mcr jbc$command
JBC$COMMAND> diag 7

Jean-FranÃ§ois

Re: Jobs stay in status starting

Wim Van den Wyngaert — Fri, 20 Jan 2006 02:51:33 GMT

JF,

Quotas are monitored. No problems. Channels idem. And I did a diag 7. Nothing changed.

Wim

Re: Jobs stay in status starting

Steve Schultz_1 — Fri, 20 Jan 2006 09:16:55 GMT

I have seen your same situation with batch jobs in a starting status. Problem was caused by someone who left their session in authorize and locked the sysuaf file. It only effected batch jobs and users were able to still log in. Once the authorize session was ended the batch jobs executed. HP says this cannot happen, but it has a few times now. Although when I try to recreate the problem, I cannot make it happen.

Steve

Re: Jobs stay in status starting

Wim Van den Wyngaert — Sat, 21 Jan 2006 09:58:18 GMT

I still have the crash dump. Will check if anyone was in sysuaf.

Wim

Re: Jobs stay in status starting

Ian Miller. — Sat, 21 Jan 2006 16:18:46 GMT

In the crash dump see what the Q manager was doing.

Re: Jobs stay in status starting

Wim Van den Wyngaert — Mon, 23 Jan 2006 02:49:57 GMT

Steve : no process was using authorize.exe. But it was a good idea.

Ian : I even restarted the queue manager (with stop/id). What do you mean with "was doing" ?

Wim