topic Re: Job with dependencies on current job in Operating System - OpenVMS

Job with dependencies on current job

roose — Tue, 15 Apr 2008 05:17:28 GMT

Hi folks,

We currently don't have 3rd party tools on our site to do our job scheduling, thus we have to rely on scripting on most of our jobs.

Currently, we need to implement a job that has dependencies on it. Previously, we submit each of the jobs in the job stream using the /after parameter. However, with this new job, the finish time of the predecessor jobs has been irregular, thus we need to create a script that will initially submit all the downstream jobs on hold (/hold), and have the current job release it after it reaches a certain stage on its execution.

I have been studying the F$GETQUI lexical, but its a bit hard to understand this particular lexical.

As an example, I have jobs A, B and C that runs on FOOBAR$BATCH. I'll submit all 3 jobs at once, but only job A will process first, and jobs B and C will be on hold. After a certain stage in job A, I'll need to release job B, which will release job C later once it finishes, while job A is still doing some other stuff.

I have this script that I have modified slightly from the F$GETQUI sample materials, but I don't understand how I can specify that I only want to check FOOBAR$BATCH, and only job B (from job A's context) will be released:

$ TEMP = F$GETQUI("")
$ QLOOP:
$ QNAME = F$GETQUI("DISPLAY_QUEUE","QUEUE_NAME","*")
$ IF QNAME .EQS. "" THEN EXIT
$ JLOOP:
$ IF NOACCESS .EQS. "" THEN GOTO QLOOP
$ JNAME = F$GETQUI("DISPLAY_JOB","JOB_NAME",,"FREEZE_CONTEXT")
$ IF JNAME .EQS. "MY_JOB"
$ THEN
$ JNUM = F$GETQUI("DISPLAY_JOB","ENTRY_NUMBER",,"FREEZE_CONTEXT")
$ JSTAT = F$GETQUI("DISPLAY_JOB","JOB_HOLDING",,"FREEZE_CONTEXT")
$ IF JSTAT THEN SET ENTRY 'JNUM'/RELEASE
$ ENDIF
$ GOTO JLOOP

Anyone can help me on this, please? Thanks!

Re: Job with dependencies on current job

Wim Van den Wyngaert — Tue, 15 Apr 2008 05:46:00 GMT

QNAME = F$GETQUI("DISPLAY_QUEUE","QUEUE_NAME","*")

must become
QNAME = F$GETQUI("DISPLAY_QUEUE","QUEUE_NAME","FOOBAR$BATCH","WILDCARD")

Wim

Re: Job with dependencies on current job

labadie_1 — Tue, 15 Apr 2008 05:55:29 GMT

Hello

Your want to synchronize and communicate between Vms processes.

You have many possibilities

- the Vms command synchronize job-name or synchronize/entry=

- use a lock

- use a logical name

- use a file in which a process writes and others read

- use a mailbox

- use a global section

- use a local event flag

Have a look at the doc
Process Communication
at
http://www.itec.suny.edu/scsys/vms/vmsdoc/72final/5841/5841pro_002.html#chap_process_communication

Re: Job with dependencies on current job

Wim Van den Wyngaert — Tue, 15 Apr 2008 06:04:46 GMT

I'm missing something "all_jobs" in your script.

You can replace the word all_jobs by "holding_jobs" and you will only get the jobs in hold. Then the jstat must no longer be tested.

Wim (agree that this is a hard-to-get lexical)

Re: Job with dependencies on current job

Jan van den Ende — Tue, 15 Apr 2008 06:15:57 GMT

Roose,

>>>
I'll submit all 3 jobs at once, but only job A will process first, and jobs B and C will be on hold.
After a certain stage in job A,
<<<
As Gerard Labadie wrote, there are many ways.
The way _I_ would do that, is to make a simple 4th job:

$ SUBMIT B /Que=.../hold
$ entry_B = $entry
$ submit C /que=.../hold/param=entry_b
$ submit A /que=.../param=entry_b

Modify A.COM: "after a certain stage" add one line:
$ set entry p1/release

Modify C.COM: Add as first line
$ SYNCHRONISE /entry='p1'

Lookup HELP SYNCHRONISE and HELP SUBMIT QUALIFIERS for the details.

hth

Proost.

Have one on em.

jpe

Re: Job with dependencies on current job

Ian Miller. — Tue, 15 Apr 2008 08:12:01 GMT

For examples on how to use F$GETQUI See VTJ #11
http://h71000.www7.hp.com/openvms/journal/index.html#getqui

and

http://dcl.openvms.org

Re: Job with dependencies on current job

roose — Tue, 15 Apr 2008 08:52:11 GMT

Thanks for the help folks! I think I got the idea now of what I need to do :)

Re: Job with dependencies on current job

John Gillings — Tue, 15 Apr 2008 22:09:21 GMT

roose,

As others have suggested, SYNCHRONIZE is the way to go to serialise jobs. For some reason it seems to not be used much, which is a bit of a surprise because it's a very useful command, especially in conjunction with the symbol $ENTRY defined after a SUBMIT or PRINT command

Here are a few hints and tips on using SYNCHRONIZE.

SYNCH against a completed, retained job will return immediately, and set $STATUS to the completion status of the job. Consider these command sequences:

$ SUBMIT/RETAIN=ALWAYS A
$ entry_A=$ENTRY

sometime later, possibly in another job:

$ SYNCHRONIZE='entry_A'
$ status_A=$STATUS
$ DELETE/ENTRY='entry_A'

SYNCHRONIZE with a non-existent entry will result in JBC-E-NOSUCHENT. To check if a particular entry exists, use

F$GETQUI("DISPLAY_ENTRY","ENTRY_NUMBER",entry)

If the entry does not exist, F$GETQUI will return a blank (and no error). If you're cautious, test:

F$GETQUI("DISPLAY_ENTRY","ENTRY_NUMBER",entry).EQS.entry

to confirm that the entry really exists.

Also note that if you know the name of your job, you don't need to do F$GETQUI loops across all queues. You can go direct to the matching entries:

$ entry=F$GETQUI("DISPLAY_ENTRY","ENTRY_NUMBER","MY_JOB")

will return the first found job with matching name. If there may be more than one, add "WILDCARD" to turn it into a loop.

Try this:

$ EntryLoop: e=F$GETQUI("DISPLAY_ENTRY","ENTRY_NUMBER",name,"WILDCARD")
$ IF e.NES.""
$ THEN
$ s=F$GETQUI("DISPLAY_ENTRY","JOB_STATUS",e,"FREEZE_CONTEXT")
$ GOSUB JobStatus's'
$ GOTO EntryLoop
$ ENDIF
$
$ JobStatus1: ! JOB_ABORTING
$ JobStatus2: ! JOB_EXECUTING
$ JobStatus8: ! JOB_INACCESSIBLE
$ JobStatus16: ! JOB_REFUSED
$ JobStatus32: ! JOB_REQUEUE
$ JobStatus64: ! JOB_RESTARTING
$ JobStatus128: ! JOB_RETAINED
$ JobStatus256: ! JOB_STARTING
$ JobStatus1024: ! JOB_SUSPENDED
$ JobStatus2048: ! JOB_PENDING
$ JobStatus4096: ! JOB_UNDEFINED
$ JobStatus8192: ! JOB_STALLED
$ JobStatus16384: ! JOB_INCOMPLETE
$ JobStatus32768: ! JOB_COMPLETING
$! do nothing
$ RETURN
$
$ JobStatus4: ! JOB_HOLDING
$ JobStatus512: ! JOB_TIMED_RELEASE
$ SET ENTRY/RELEASE 'e'
$ RETURN

FWIW, I recently confirmed with OpenVMS engineering that a job status will only ever have a single bit set, so the above DCL "CASE" statement will work correctly.

Another curiosity is that SYNCHRONIZE will work against a PRINT job as well as batch job.

As well as serialising jobs, you can use synchronize to parallelise. Break the job up into a number of tasks, which can be SUBMITted as independent jobs, possibly across a cluster. Use /RETAIN=ALWAYS, and accumulate a list of entry numbers. Once they're all submitted, sit in a loop SYNCHRONIZEing against the entries, and cleaning up the completed ones. You can also copy the log files to SYS$OUTPUT so they're all in one place.

Re: Job with dependencies on current job

Wim Van den Wyngaert — Wed, 16 Apr 2008 05:37:54 GMT

Everyone seems to forget :
After a certain stage in job A, I'll need to release job B.

So the job is not finished yet and thus sync will not do the work.

Wim

Re: Job with dependencies on current job

Wim Van den Wyngaert — Wed, 16 Apr 2008 05:40:23 GMT

F$GETQUI("DISPLAY_ENTRY","ENTRY_NUMBER","MY_JOB")

Tried it and it doesn't return anything. With the entry number it does.

John ?

Wim

Re: Job with dependencies on current job

Wim Van den Wyngaert — Wed, 16 Apr 2008 07:07:33 GMT

Solved. It works when it's your job, t.i. the same username.

Still puzzled that you may pass the job name as an entry number.

Wim

Re: Job with dependencies on current job

Jan van den Ende — Wed, 16 Apr 2008 07:09:46 GMT

@Wim:

>>>
Everyone seems to forget :
After a certain stage in job A, I'll need to release job B.

So the job is not finished yet and thus sync will not do the work.
<<<

Eh?
Please re-read my code snippet. It SYNCHRONISEs on the to-be-completed job, but, as requested, "at a certain stage", it SET ENTRY=../RELEASE for the other one!

fwiw.

Proost.

Have one on me.

jpe

Re: Job with dependencies on current job

Wim Van den Wyngaert — Wed, 16 Apr 2008 07:45:07 GMT

No longer puzzled : it's documented.

The object-id argument specifies either the name or the number of an object (for example, a specific queue name, job name, or form number) about which F$GETQUI is to return information.

@Jan : Sorry. Overread that.

Wim (wishing for Google performance at HP forums)