Migrating queue entries from one device to another

[Jon Pinkley]

We are migrating our storage from HSZ DK devices to EVA, and in the process reinitializing our target devices with more consistent clustersizes (all will be powers of 2 going forward). We are going to be moving our datacenter, and to provide a fallback position, we are using HBVS to keep the data current on both the EVA and HSZ storage. Only the EVA storage will be moving.

I have a command procedure that migrates the disk, and all our software uses device logical names, so as a device moves, we only need to update our mount_disks.com with the new devices.

However, we have many queues, both print and batch, and most printers are at our branch locations. It is normal for us to have thousands of print jobs queued, but not printed, due to printer stalled or stock mismatch. Under normal conditions, we have fewer batch jobs, but there are jobs that are time released or executing at all times.

Queue entries store the files using physical device names and (I am guessing here) file id. So if the physical name of the device changes, existing queued entries will fail if they are allowed to start.

This is one advantage of using HBVS, as entries will then be referencing the DSA virtual unit, and even if the underlying physical device changes, the queue entries are unaware of this change. LD devices could also be used to provide device independence if shadowing is not an option, however shadowing has the advantage that the move can be done without ever taking the virtual unit offline.

However, we are in the situation of moving from DK devices to DSA devices, so I still have to deal with migrating entries from one device to another.

I am wondering what tools are available to help automate this.

What I am considering at the present time are modifications to tools that have been developed to recreate the queue database, such as the FIXQUE.COM tool from the freeware. This would be used to create a command procedure, and that would be edited to extract only entries that reference the physical device being changed, those queues would then be stopped, the device moved via backup/image to the DSA device, then the original entries deleted, and the entries requeued using the new physical device name.

Is there a better approach to this problem?

[Thomas Ritter]

We have done this a few times. Our approach has been to run the new and old in parallel and move usually just newly created log files to the new disks.
Our command procedures are are located in only a few places. The input command procedures on the old disks were kept until such times as no more batch jobs referenced them. If I recall correctly it took a few weeks before all new batch entries were referencing the new disks.

In another migration the new disks had to replace the old disks. We wrote command procedures to resubmit the old jobs. This, with a lot of manual checking, worked okay.

Our applications use VMS logical names really well. If there is hard coding occuring then migration will require careful checking.

[Verne Britton]

I may be setting myself up for big troubles down the road, but about 3 years ago I made all my data disks be one-member shadow sets, to get everything on a DSAxxxx device name.

Doesn't help you much today though :-)


Verne

[Jess Goodman · Accepted solution]

Jon,

Per your request I am advertising my DISPLAY_JOBS.COM in this thread. This procedure can be used to make reSUBMIT commands for batch jobs on all queues, or for a specific [wildcarded] queue name, or for the [named] batch entries owned by the current user.

You could run it and create a command procedure with all the SUBMITs needed to repopulate your batch queues; edit the output to modify the device name of the batch command procedures; delete all batch jobs; and then run the procedure. Sorry, it will not handle print jobs.

I have a attached a newer version to this post. It is better documented and will output help with the command:
$ @DISPLAY_JOBS ?

I fixed a couple of minor typos and a bug with batch jobs for multiple files.

[Jon Pinkley]

The original submit command as submitted by user JON Alpha VMS 7.3-2

$ submit/hold/cpu=400-01:23:45.67 scr:display_jobs.com/que=sigma$oper/notify/noprint/log=scr/name=test_of_display_jobs/prio=200 /after=31-dec-2007/note="A test of the capabilities of Jess Goodman's ""DISPLAY_JOBS.COM"" compared with others" /retain=until=1-jan-2008/characteristics=(legal,pcl)/style=expanded/wsdef=40000/wsext=1763328/wsquo=100000 /param=("","","","''f$fao("!!!200*")'")
Job TEST_OF_DISPLAY_JOBS (queue SIGMA$OPER, entry 2039) holding
$


Display from show queue /full (what FIXQUUE.COM uses as its input) and show entry /full

$ sho que/ful sigma$oper
Batch queue SIGMA$OPER, idle, on SIGMA::
/BASE_PRIORITY=4 /JOB_LIMIT=15 /OWNER=[SYSTEM] /PROTECTION=(S:RSMD,O,G,W)

Entry Jobname Username Status
----- ------- -------- ------
2039 TEST_OF_DISPLAY_JOBS
JON Holding
Submitted 3-AUG-2007 04:11:09.06 /CHAR=(4,11) /CPU=22-F-B-1859 01:0 /KEEP /LOG=$4$DKA200:[USERS.][JON.SCRATCH].LOG;
/NOTE="A test of the capabilities of Jess Goodman's "DISPLAY_JOBS.COM" compared with others" /NOTIFY /PARAM=("","","",
"!4444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
/NOPRINT /PRIORITY=200 /RETAIN=UNTIL="1-JAN-2008 00:00" /WSDEFAULT=40000 /WSEXTENT=1763328 /WSQUOTA=100000
File: _$4$DKA200:[USERS.JON.SCRATCH]DISPLAY_JOBS.COM;1
$ sho ent/ful 2039
Entry Jobname Username Blocks Status
----- ------- -------- ------ ------
2039 TEST_OF_DISPLAY_JOBS
JON Holding
On idle batch queue SIGMA$OPER
Submitted 3-AUG-2007 04:11:09.06 /CHAR=(4,11) /CPU=22-F-B-1859 01:0 /KEEP /LOG=$4$DKA200:[USERS.][JON.SCRATCH].LOG;
/NOTE="A test of the capabilities of Jess Goodman's "DISPLAY_JOBS.COM" compared with others" /NOTIFY /PARAM=("","","",
"!4444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
/NOPRINT /PRIORITY=200 /RETAIN=UNTIL="1-JAN-2008 00:00" /WSDEFAULT=40000 /WSEXTENT=1763328 /WSQUOTA=100000
File: _$4$DKA200:[USERS.JON.SCRATCH]DISPLAY_JOBS.COM;1
$

Some things to note: Look at the CPU limit (this is a bug documented in Jess Goodman's DISPLAY_JOBS.COM). Also note that if both /after and /hold are specified, the after time is lost in the show queue/full and show entry/full output. Also, long parameter values are truncated.

Following is what FIXQUE.COM recreates from the SHOW QUEUE/FULL output

$SUBMIT/NAME="TEST_OF_DISPLAY_JOBS"/HOLD/USER=JON/CHAR=(4,11)-
/CPU=22-F-B-1859 01:0/KEEP/LOG=$4$DKA200:[USERS.][JON.SCRATCH].LOG;-
/NOTE="A test of the capabilities of Jess Goodman's "DISPLAY_JOBS.COM" compared with others"-
/NOTIFY-
/PARAM=("","","","!44444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
44444444444-
/NOPRINT/PRIORITY=200/RETAIN=UNTIL="1-JAN-2008 00:00"/WSDEFAULT=40000-
/WSEXTENT=1763328/WSQUOTA=100000/QUEUE=SIGMA$OPER -
_$4$DKA200:[USERS.JON.SCRATCH]DISPLAY_JOBS.COM;1

Specifically note that it isn't a valid command.

$SUBMIT/NAME="TEST_OF_DISPLAY_JOBS"/HOLD/USER=JON/CHAR=(4,11)-
/CPU=22-F-B-1859 01:0/KEEP/LOG=$4$DKA200:[USERS.][JON.SCRATCH].LOG;-
/NOTE="A test of the capabilities of Jess Goodman's "DISPLAY_JOBS.COM" compared with others"-
/NOTIFY-
/PARAM=("","","","!44444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
44444444444-
%DCL-W-NOPAREN, value improperly delimited - supply parenthesis
\!4444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444-\
/NOPRINT/PRIORITY=200/RETAIN=UNTIL="1-JAN-2008 00:00"/WSDEFAULT=40000-
%DCL-W-NOCOMD, no command on line - reenter with alphabetic first character
/WSEXTENT=1763328/WSQUOTA=100000/QUEUE=SIGMA$OPER -
_$4$DKA200:[USERS.JON.SCRATCH]DISPLAY_JOBS.COM;1


This is due to the formating of the CPU limit done by show queue/full and the truncation of the parameter statement.

Also, since the after time does not show up in the output of show queue/full, FIXQUE has no way to regenerate the /AFTER in the sumbit command it generates.

So, is there a better way to recreate the recreate a command that will recreate the entry as closely as possible? I think so, and the answer is to use the $GETQUI, available in DCL via the F$GETQUI lexical function.

In thread http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1144436 sdi-1 (Jose Baars, mostly) provides a command procedure dumpqdb.com that was written to recover from corrupted queue database. As such, it was written to deal with the set of features that were needed to get them back online as quickly as possible, so it doesn't try to do as much as FIXQUE does.

We have too many queues for his dumpqdb.com procedure to work, I get

%DCL-W-SYMOVF, no room for symbol definitions - delete some symbols

errors.

It will probably work correctly for 99+% of VMS sites, but we are in the small minority that have many queues and many entries.

After I had tried the original version of dumpqdb and reported my findings about FIXQUE and dumpqdb, Jess Goodman posted his DISPLAY_JOBS.COM in that thread, I liked what I saw and asked him to make an entry here. He updated his procedure to add some documentation, and atteached it to the above thread as well as here.

Here is the output of DISPLAY_JOBS.COM for this job, and it executes without error. Note the wrapping is being done by the ITRC display.

$ @display_jobs/out=test.com "" 2039
$ @test.com
$ SUBMIT _$4$DKA200:[USERS.JON.SCRATCH]DISPLAY_JOBS.COM;1 -
/AFTER=31-DEC-2007:00:00:00.00 -
/CHARAC=(4,11) -
/CPUTIME=400-01:23:45.67 -
/HOLD -
/LOG=$4$DKA200:[USERS.][JON.SCRATCH].LOG; -
/NOPRINT -
/NAME="TEST_OF_DISPLAY_JOBS" -
/NOTE="A test of the capabilities of Jess Goodman's ""DISPLAY_JOBS.COM"" compared with others" -
/NOTIFY -
/PARAM=("","","","!44444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444") -
/PRIORITY=200 -
/RETAIN=UNTIL=1-JAN-2008:00:00:00.00 -
/WSDEFAULT=40000 -
/WSEXTENT=1763328 -
/WSQUOTA=100000 -
/QUEUE=SIGMA$OPER:
Job TEST_OF_DISPLAY_JOBS (queue SIGMA$OPER, entry 2729) holding
$ sho ent/ful 2729
Entry Jobname Username Blocks Status
----- ------- -------- ------ ------
2729 TEST_OF_DISPLAY_JOBS
JON Holding
On idle batch queue SIGMA$OPER
Submitted 3-AUG-2007 04:33:08.61 /CHAR=(4,11) /CPU=22-F-B-1859 01:0 /KEEP /LOG=$4$DKA200:[USERS.][JON.SCRATCH].LOG;
/NOTE="A test of the capabilities of Jess Goodman's "DISPLAY_JOBS.COM" compared with others" /NOTIFY /PARAM=("","","",
"!4444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444444
/NOPRINT /PRIORITY=200 /RETAIN=UNTIL="1-JAN-2008 00:00" /WSDEFAULT=40000 /WSEXTENT=1763328 /WSQUOTA=100000
File: _$4$DKA200:[USERS.JON.SCRATCH]DISPLAY_JOBS.COM;1
$


In the thread referenced above, Jess Goodman says: "I've attached my DISPLAY_JOBS.COM which is an extremely robust procedure that uses F$GETQUI to recreate SUBMIT commands for batch jobs."

Extremely robust is a big claim. After testing it, I have to agree.

Good work Jess! And thanks for sharing it.

Jon

P.S. Jess, how did you find the bug in the CPU limit? If you discovered it by testing, you did a good job of testing. And the workaround was quite clever.

[Ian Miller.]

Jess,
can you submit the updated version to dcl.openvms.org so it can be available to all.

[Jess Goodman]

Jon,

Thanks for the kind words.

I discovered the high CPU limit bug in F$GETQUI and SHOW QUEUE/ENTRY while testing DISPLAY_JOBS.COM and my SUBMIT.FOR (a subroutine that takes as its argument a SUBMIT command line string). I started a thread discussing the bug at the time:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1073701

What was even trickier to handle was the DCL limit of 256 bytes as the max. record size of a .COM file prior to VMS 7.3-2 when eXtended DCL came out.

A quoted /NOTE or /PARAMETER string can be longer than 256 bytes just by itself when you count the surrounding quotes and any inner doubled-qoutes. It was not easy to find a way to break up a quoted string with a continuation line but I found a method using an obscure DCL "feature".

I will eventually submit DISPLAY_JOBS.COM to dcl.openvms.org but I was thinking of first adding two other features to it.

* An option to output DELETE/ENTRY commands prior to each SUBMIT command.
* An option to submit the most recent version of the batch .COM file(s) instead of the version used by the current job.

Putting these two options together you could use this after editing a batch .COM file as an easy way to resubmit existing jobs to use your new version.

However, I can not think of an intuitive way of requesting these two options on the @DISPLAY_JOBS command line. Any suggestions?