topic Re: jobs sitting in queues in Operating System - OpenVMS

jobs sitting in queues

Lucinda_1 — Tue, 18 Oct 2005 10:27:10 GMT

OpenVMS 7.3-2 tri-host cluster. 2)es45 production nodes/ 1)ds25 develop. node All patches are current.
"jobs" print and BATCH will sit in the que and do nothing. The log files indicate nothing, the job just simply stops and stays there. No adverse i/o on system, nothing. The job will sit there until it is killed. Run the job again, all is fine. I cannot seem to pin down one thing that is a constant, ie user, particular job, or even particular application area. Seems it has something to do with the queues, files system, or someting that is "overall" involved. But again, no errors to help lead to a solution.

Re: jobs sitting in queues

Ian McKerracher_1 — Tue, 18 Oct 2005 10:43:08 GMT

Hello Lucinda,

I would be tempted to check again that your patches are current as there has been a similar problem in the past and an ECO was released.

Regards,

Ian

Re: jobs sitting in queues

Daniel Fernandez Illan — Tue, 18 Oct 2005 11:54:33 GMT

Hi Lucinda.
Are you modified recently node names of your environment?
In this case you must check definition of queues (qualifier /ON)
Saludos.
Daniel.

Re: jobs sitting in queues

Volker Halle — Tue, 18 Oct 2005 12:11:42 GMT

Lucinda,

if this problem happens, what is the status of the jobs and the status of the queues, these jobs are in ?

Any errors in OPERATOR.LOG ?

Volker.

Re: jobs sitting in queues

Lucinda_1 — Tue, 18 Oct 2005 12:47:20 GMT

sorting thru the few patches not installed. Not finding any relevant
no modifications in node names. no errors in any log file. there is one job log, additionally there is a log kept in the users directory, no errors found. I have submitted this problem to Compaq as well as to the company that provides support for our file system, no help.

Re: jobs sitting in queues

Arch_Muthiah — Tue, 18 Oct 2005 13:06:30 GMT

Hi Perry,

How you configured your printer ?

Configured as LPD printer or telnet printer?

In anycase, if you send output of

$ SHOW QUEUE/MANAGER ---> will make sure your queue manager is running

$ SHOW QUEUE ---> list the print job submitted and the status of each print job and you can see to which node your TCPIP$LPD queue has been mounted.

If you see the result of hte above command, you can easily figure out the problem, or send to us.

In otherway, I would suggest you to do this trials...

Reconfigure your LPD printer..
Before you configure TCPIP$LPD printer, your queue manager should start and running...

$ start/queue/manger if your queue manger not running already.

reconfigure LPD printer using sys$manager:tcpip$config.com procedure.
You knew LPD printer uses 515 port#.

now submit any print job,
if $SHOW QUEUE still hanging with pending state, please send the output of

$SHOW QUEUE

In other way, (I would not suggest you to have the Telnet printer instead of LPD configured printer)
If you would like to test that you can communicate with your printer using TELNET,
just try to do this trials

$ init/queue/start/process = TCPIP$TELNETsym
/on = "IP-address-of-your-printer:port#"
symbol_for_your_printer

this will create/initialize a new telnet printer.

now ping to that telnet printer using
$ TELNET IP_Address_of_your_printer port#

Note: Sysadmin knows the assigned port number for this printer, mostly 9100 will be the port# for HP branded printer.

Archunan

Re: jobs sitting in queues

Lucinda_1 — Tue, 18 Oct 2005 14:20:22 GMT

this is not just a printer problem, it happens in job queues as well, i have lpd, lat, and TCPIP$TELNETSYM printers.

Re: jobs sitting in queues

Ian Miller. — Tue, 18 Oct 2005 14:53:16 GMT

the output of the show commands previously would be interesting. Also is the queue journal file large?

Re: jobs sitting in queues

Lucinda_1 — Tue, 18 Oct 2005 15:20:51 GMT

que journal file?

Re: jobs sitting in queues

Lucinda_1 — Tue, 18 Oct 2005 15:24:12 GMT

i am downloading patches, and I have to schedule install. I will try the printer directions and let you know the results. The tough part is that it doesnt always hang.

Re: jobs sitting in queues

Ian Miller. — Tue, 18 Oct 2005 16:32:37 GMT

queue journal file
SYS$QUEUE_MANAGER.QMAN$JOURNAL

There has been past bugs where this would get large and there was trouble.

is the queue on the same node as the queue manager is running or another node. If another node then could there more an issue with the cluster communications between the nodes?
(queues use ipc)

Re: jobs sitting in queues

Robert_Boyd — Wed, 19 Oct 2005 09:59:30 GMT

Lucinda,

Another troubleshooting approach on this problem -- when a job is hanging, try enabling operator messages at your session:

$ REPLY/ENABLE

Then issue

$ REPLY/STATUS

And see what if anything you get.

Also, if you have operator logging enabled then you might look in SYS$MANAGER:OPERATOR.LOG or whatever file OPC$LOGFILE_NAME is pointing at.

Also, if you see a job stuck like that you might go into ANALYZE/SYSTEM and do a
SHOW PROCESS/ID=/CHANNEL
for the pid of the hung job and see what I/O channels are open and busy (if any).

Robert

Re: jobs sitting in queues

Art Wiens — Wed, 19 Oct 2005 14:37:34 GMT

I have had trouble where the system time on the cluster member running the queue manager was several minutes out from cluster members that were doing the submitting/printing. Jobs that were holding for a particular time did not start as expected, because it wasn't time yet according to the queue manager.

Art

Re: jobs sitting in queues

John Yu_1 — Sat, 22 Oct 2005 07:27:30 GMT

Might be process limits or quotas?

Re: jobs sitting in queues

Lawrence Czlapinski — Mon, 24 Oct 2005 13:24:21 GMT

Lucinda:
1. For the printers,
$show log/sys lat$symparameter
We had problems where sometimes a symbiont would hang. When a printer queue hangs, you could check whether the printers are using the same symbiont. You might try lowering the value of LAT$SYMPARAMETER.

$! * Define the maximum number of printer symbionts for LAT queues
$! ( 1 means new symbiont each time a queue stops ...
$! ( 4 is an o.k. alternative if performance suffers [default 16?]
$ define/sys/exec lat$symparameter "1"
2. Are all three nodes using the same queue manager? If they aren't, we have seen the journal file grow quite large on one of our nodes and we had assorted queue problems.
3. Since you are having problems with both printers and batches, it might be a problem with the queue manager journal file. You could check to see whether it has grown quite large.
4. I'm assuming your CPUs are not running at 100% on the node(s) that have the batch queues since that could keep low priority batch jobs from getting a time slice.
Lawrence

Re: jobs sitting in queues

Lucinda_1 — Mon, 24 Oct 2005 15:14:13 GMT

The qman$journal is not large. I did check the LAT$SYMPARAMETER, it is default 16 on both nodes in question. CPU is fine. I did find an ask the wizard that also included
DEFINE/SYSTEM tcpip$TELNETSYM_STREAMS 16
I have now defined on both systems. I have scheduled patches for Nov 11, but in reading I did not find any that eluded to my problems. All three nodes use the same que manager. I have many nightly jobs that run under the "operator" user account, at times they sit in the que (there are other jobs ran by other users as well) I have bumped up the user quotas to equal the system. (it is effectively a system account anyway). It is definitely not a time issue as many of the jobs are submitted by the user to run now. The problem happens on both of the production nodes, the que manager runs on just one.

Re: jobs sitting in queues

Lawrence Czlapinski — Mon, 24 Oct 2005 16:07:41 GMT

Lucinda:
1. Having a separate batch queue might help the Operator jobs.
2. You could try a SHOW PROC/CONTINOUS/ID=id_number to see if the process is looping or waiting for something.
Lawrence