topic Re: Reset Entry numbers in Operating System - OpenVMS

Reset Entry numbers

owilliams — Thu, 12 Jul 2007 13:48:59 GMT

A runaway process caused our entry numbers to jump from low numbers to very high numbers. Is there a way to reset the entry numbers to a lower number? Thanks for any assistance.

Re: Reset Entry numbers

Dale A. Marcy — Thu, 12 Jul 2007 14:19:37 GMT

Here is a similar thread that might answer your question:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=1097922

Re: Reset Entry numbers

Jan van den Ende — Thu, 12 Jul 2007 14:32:48 GMT

owilliams,

3 possible answers.

1. Just wait, they will roll over and then start a 1 again.

2. If you _REALLY_ want to get rid of the large numbers, make an inventory of all queues (and, if really needed, the jobs).
Stop the queue manager, delete the queue files (well, better rename them away for backup!) and create a new set of que manager files.
Recreate your queus, and resubmit your jobs.

3. Use the queue reset commands. I do not have them at hand, but will look them up if nobody beats me to it. I do remember that there were some 'interesting' details though.

Finally: both with 2. and with 3. : be prepared for a repeating issue.

We have also had it happen, we have done the reset once, and next month it was there again.

And, below the line:
What problem are you trying to solve?
A little nuisance with big numbers in occasional cases?
Better just learn to live with it.

Proost.

Have one on me.

jpe

Re: Reset Entry numbers

owilliams — Thu, 12 Jul 2007 15:03:43 GMT

Is there a way to manually reset the numbers without playing with the queues?

Re: Reset Entry numbers

Hoff — Thu, 12 Jul 2007 15:04:17 GMT

The classic HP response is that the job controller entry number is an opaque longword, and no further assumptions about the value should be made.

On more recent releases, there was a change made to the queue manager to try to compress the range of numbers downward where feasible. Off the top, I don't remember the ECO and release details that were involved there.

On older releases, the classic approaches were to force-roll the entries back to the low range (basically wrapping the counters), or to brute-force re-create the entries and the whole of the queue database with a start /queue /manager /new.

Darren Boyle posted the following example of wrapping the entries a number of years back, to seek to force the wrapping of the value back to its low range:

$ loop:
$ set noon
$ submit/hold login.com
$ job_no = $entry
$ delete/entry=job_no1
$ submit/hold login.com
$ job_no1 = $entry
$ dele/entry= job_no
$ goto loop

Re: Reset Entry numbers

owilliams — Thu, 12 Jul 2007 15:42:25 GMT

Is there any downside to forcing the wrapping of the value back to a lower range of numbers?

Is there any evidence of a high number being a detriment to the system in any way?

Re: Reset Entry numbers

Robert Brooks_1 — Thu, 12 Jul 2007 15:59:36 GMT

Is there any evidence of a high number being a detriment to the system in any way?

--

Only if there is poorly-written software that expects an entry number to be less than a certain value.

The VMS Operating system itself does not have that flaw; your local applications may have that problem.

-- Rob

Re: Reset Entry numbers

owilliams — Thu, 12 Jul 2007 16:19:48 GMT

I didn't think so but we have some nervous folks at work that thought it might be a problem.

Re: Reset Entry numbers

Hoff — Thu, 12 Jul 2007 16:45:52 GMT

The appropriate response to >>> I didn't think so but we have some nervous folks at work that thought it might be a problem<<< would seem to involve testing; to run series of big entry numbers into play, and to simply try it. This testing while you're awake, aware, prepared and ready and not otherwise wondering what just happened when a key application tipped over.

Re: Reset Entry numbers

Jiri_5 — Fri, 13 Jul 2007 03:37:45 GMT

You can stop old queue manager and start new with param (/new). After that you must create all queues which you need. on this node. It reset you batch number to number 0

Re: Reset Entry numbers

The Brit — Fri, 13 Jul 2007 07:20:02 GMT

I know of at least one major application which had a problem with this. Their queue administration module assumed that the entry number would be 4 digits or less. To make things worse, when the entry#'s jumped from 4 to 7 digits, the module displayed the first 4, making it virtually unusable. Although the App Vendor is now peddling a new product to replace the old app, there are probably still many clients using the old app.

Re: Reset Entry numbers

SDIH1 — Tue, 24 Jul 2007 13:18:03 GMT

I ran in to this a year ago, producing entry numbers that in some apps came up as negative numbers :-(.
On the freeware cd's there was a utility called fixque ( I think by Keith Parris) that dumped the queue database, and parsed out the output of show queue/full etc. Problem was that it couldn't handle the large entrynumbers either :-). So I wrote a dumpqdb.com based on getqui
that produced an output file with all the commands necessary to create a new queue database.

Couple of things missing : characteristics, we don't use it. ACL's on queue, ditto. /new switch to the start/que/manager command. Probably more, there are a lot of things that we don't use, but are possible.

To use it, create the output file by @dumqdb/out=w.w, then carefully compare w.w to your queues and entries, edit in omissions or additions, add the /new switch to the start/que/manager command.
Then stop the queumanager, backup, and then rename or delete all queue database files, as they probably have grown beyond any reasonable size and run w.w. You should (almost, right in three tries is my sad motto) be back to normal.

This was all written as an emergency procedure, and it did work 99% in our situation, but this might be different in yours.

Re: Reset Entry numbers

Ian Miller. — Tue, 24 Jul 2007 14:09:44 GMT

Jose,
can you post that to dcl.openvms.org as well as it looks like it could be useful for somebody.

Re: Reset Entry numbers

Dean McGorrill — Tue, 24 Jul 2007 16:28:42 GMT

I suppose one could locate the longword cell
where the entry number is kept and stick a
low value in it. assuming all the jobs are
high numbers they wouldn't collide. (I've
never tried it though)

Re: Reset Entry numbers

David Jones_21 — Tue, 24 Jul 2007 19:13:08 GMT

I like having 3-digt entry numbers just for asthetic reasons. Since we already have command procedures to re-create all queues from scratch, it isn't that big a deal to reset the queue database.

Samba seems to assume job numbers below 10,000, but I think Unix job numbers are per-queue rather than global as in VMS.

Re: Reset Entry numbers

John Gillings — Tue, 24 Jul 2007 19:41:18 GMT

A brute force approach to resetting entry numbers. No special privileges required.

The attached procedure submits itself /HOLD, then immediately deletes the entry. It then compares the entry number against a parameter, and if greater, repeats.

If you're in a high range, execute the procedure with a parameter of (say) 10000. It will repeat until the entry number wraps. Provided the number you specify is more than the number of concurrent entry numbers you have, the loop will eventually complete. With a high enough threshold, you could safely run the procedure regularly to make sure your entry numbers don't exceed your limit.

Once the number has wrapped, just wait for the high numbered entries to complete (or resubmit them), and the queue manager should reset itself into the lower range.

$ IF p1.EQS."" THEN EXIT
$ self=F$PARSE(";",F$ENVIRONMENT("PROCEDURE"))
$ loop:
$ SUBMIT/HOLD 'self'
$ DELETEX/ENTRY='$ENTRY'
$ IF F$INTEGER($ENTRY).GT.F$INTEGER(p1) THEN GOTO loop
$ EXIT

Re: Reset Entry numbers

Jon Pinkley — Sun, 29 Jul 2007 04:58:08 GMT

Here are my experiences with FIXQUE and dumpqdb.

We have a relatively complex queue database, (1088 total queues. 667 batch queues, of which 153 are generic. Queues with text descriptions, acls, characteristics, many forms and multiple stock types)

The point is, just because something doesn't work for us, does not mean it will not work for you.

----------------------------------

The version of FIXQUE I tested is a modified version that was posted to comp.os.vms to allow it to work better with larger queue entry numbers.

FIXQUE evidently originated in the Colorado Support Center, I've seen versions with Keith Parris and Mark Jilson's comments, so I don't know how it evolved, but it is quite old and probably has the work of many people in it.

FIXQUE works by creating a listing using show queue commands, and parsing the output. This reason alone is enough to make it impossible to recreate an arbitrary set of queue entries. The thing that caused it to break at our site is an entry that has long quoted parameter values, and these get truncated in the output from show queue/full. Since FIXQUE gets its information from the listing file, it cannot reproduce the original entry, although it doesn't detect that the value has been truncated.

FIXQUE does do an admirable job given the limitations of the implementation, i.e. recreating the "state" from only a listing, and doing it using only software on a standard VMS system, i.e. DCL and the standard utilities.

FIXQUE creates a command file that will recreate forms, characteristics, queues (both generic and execution), acls on queues, and the entries that were in the queues at the time it was run. For jobs that are currently executing, it generates a SUBMIT /HOLD, since these jobs should be manually verified before allowing them to start over.

It also gives warnings about jobs that reference files that do not exist in any directory.

------------------------------------

dumpqdb (see sdi-1 (Jose Baars, mostly) entry in this thread dated Jul 24, 2007 18:18:03 GMT) command file is attached to entry.

dumpqdb (stated in the note) was written for a specific situation, and as such does not attempt to be as general purpose as FIXQUE.

However, it was written later than FIXQUE, and uses f$getqui to get the information to recreate the queues, which is much less likely to break in the future that attempting to parse the output of a listing.

I was happy to see that someone had used the more direct approach to gathering the queue information with f$getqui instead of using the more limited output of show queue.

I am glad it was provided, it definitely has potential, and is a good example of what can be done with f$getqui. It is much smaller than fixque.

The first time I ran the procedure, I got

$ @ROOT$USERS:[JON.SCRATCH]DUMPQDB.COM;1/out=scr:w.w
%DCL-W-IVVERB, unrecognized command verb - check validity and spelling
\SAY\
%DCL-W-IVVERB, unrecognized command verb - check validity and spelling
\SAY\
%NONAME-F-NOMSG, Message number 00000004
$

I entered

$ say :== write sys$output

and then it got further, but died when it got to a print queue that was defined like this:

$ show queue /full b00acc1
Printer queue B00ACC1, idle, on SIGMA::"acc-4000tn-1:9100", mounted form DEFAULT

/AUTOSTART_ON=(SIGMA::"acc-4000tn-1:9100",OMEGA::"acc-4000tn-1:9100") /BASE_PRIORITY=4 /DEFAULT=(FORM=DEFAULT) /LIBRARY=LEXLZ
Lowercase /OWNER=[SYSTEM] /PROCESSOR=UCX$TELNETSYM /PROTECTION=(S:M,O:D,G:R,W:S) /SCHEDULE=(NOSIZE)
(IDENTIFIER=B00MEM,ACCESS=MANAGE)
$

$ @ROOT$USERS:[JON.SCRATCH]DUMPQDB.COM;1/out=scr:w.w
looking at queue B00ACC1
%DCL-W-IVCHAR, invalid numeric value - check for invalid digits
\4000TN\
%NONAME-F-NOMSG, Message number 00000004
$

The cause of the confusion is the quoted values, and the way DCL parses quoted strings.

Specifically, the line

$ autonode = f$getqui("display_queue","autostart_on",qname,"freeze_context")

works, but what is returned in this case has embedded quoted strings.

$ show symbol autonode
AUTONODE = "SIGMA::"acc-4000tn-1:9100",OMEGA::"acc-4000tn-1:9100""

Then autonode is used in the following:

$ write sys$error "queue ''qname' has autostart set to ''autostart', autonode = ''autonode', host=''host', node = ''node'"

And when DCL substitution occurs, the following happens:

$ write sys$error "queue B00ACC1 has autostart set to TRUE, autonode = SIGMA::"acc-4000tn-1:9100",OMEGA::"acc-4000tn-1:9100", host="
acc-4000tn-1:9100", node = SIGMA"
%DCL-W-IVCHAR, invalid numeric value - check for invalid digits
\4000TN\
%NONAME-F-NOMSG, Message number 00000004
$

I wish f$edit had a built in edit rule to process a string so it could be used in a quoted DCL string, e.g. processing the single quotes and double quotes so they will work identically when enclosed in a double quoted string. I don't know of an easy way to do this in DCL.

For example: (non-implemented)

autonode_for_quoting = f$edit(autonode,"QUOTE_FIXUP") ! this does not exist

would result in

$ show symbol autonode_for_quote
AUTONODE_FOR_QUOTE = "SIGMA::""acc-4000tn-1:9100"",OMEGA::""acc-4000tn-1:9100"""

This is where I stopped the testing of dumpqdb, as to get it to work with all the features we are using would be harder than to fix fixque to use f$getqui for the entry parameters.

------------------------------------

The following is probably obvious, but needs saying:

Running any of the tools to recreate the queue database on an active system is somewhat equivalent to backup/ignore=interlock, i.e. the state of the queue can be changing underneath the software that is attempting to reproduce it.

If there are jobs that are going to synchronize on or release other entries using the entry numbers, any software that recreates the database is going to break those.

My point is that if there are a lot of queue entries in your system at the time you recreate it, you are much more likely to have problems than if there are none, and you are just recreating the queues, forms, characteristics etc.

Re: Reset Entry numbers

Art Wiens — Sun, 29 Jul 2007 10:15:42 GMT

There must have been some sort of lunar cycle ... we also had a runaway train this week! Instead of submitting itself once the next day and carry on with it's task, a coding error looped it back to the submit. It managed to do this over 148,000 times before someone remarked "the system's a bit slow isn't it?". Just to note how the entry numbers seemed to go - it looks like it counts up to 9999, increments the million's field and goes again ie. 1000000 - 1009999. I'm not sure if it finally gave out or when I killed the queue_manager and job_cocntrol pids, but it made it up to/over 14,000,000. Yes it was drastic to kill but the queue manager was solid compute and had crashed on it's own 21 times. No queue related commands were completing, no jobs were executing. I had to modify SYS$COMMON:[SYS$STARTUP]VMS$CONFIG-050_JOBCTL.COM to include a larger value for /page_file=3000000 because it was dying with :

%QMAN-F-ALLOCMEM, error allocating virtual memory
-LIB-F-INSVIRMEM, insufficient virtual memory
%JBC-E-QMANDEL, unexpected queue manager process termination

Once the queue manager was running again I saw what had happened to the entry numbers - what a mess! I was able to produce a listing of that queue and it's 148,000+ entries (all waiting to "go" at 22:10!!), learned a keyboard macro to delete the entry, EVE/TPU was able to replay it over 148,000 times (!) and I executed it. Took about half an hour to delete them all. Phew!

Not a very good friday on sysadmin day!!

http://www.sysadminday.com/

Cheers,
Art

ps. the "coding error" was NOT mine, I just had to clean it up ;-)

Re: Reset Entry numbers

SDIH1 — Tue, 31 Jul 2007 14:21:55 GMT

Hi,

I tried to overcome the fixup quote thing, attached an maybe improved procedure.
It handles the specific example of the autostart_on nodes, as far as I quickly tested.

In our case the qmanager journal file had eaten up all disk space, so a backup of the queuemanager files was not desirable or even possible. In our case we had to use a tool like this.

Re: Reset Entry numbers

owilliams — Tue, 31 Jul 2007 14:29:29 GMT

Thanks all, you have been extremely helpful.