How to limit # of user jobs executing in batch queue

[EdgarZamora_1]

Hello esteemed colleagues. Maybe I'm missing something simple here but it's not coming to me. I would like to limit the number of jobs each user can have "executing" in a batch queue. For example, I'd like to have sys$batch have a job limit of 6, but I don't want one user hogging the queue and running 6 jobs simultaneously. I'd like to limit job execution to one per user. They can submit as many jobs in the queue as they want, but only one per user will run at a time.

I don't want to maintain separate queues per user. The user accounts change frequently. The users are locked in a third-party menu which I don't have much control over (can't make application changes).

I also don't want to make changes in UAF (maxacctjobs, etc.) since I don't want to limit their number of logins, subprocesses, etc.

Any ideas would be much appreciated.

[Jan van den Ende · Accepted solution]

Edgar,

I do not know straight away of something simple.

But you can easily rool your own in SYLOGIN.

If mode eqs batch;
F4GETQUI for the user;
If found & NOT this_job, get entry nr, &
Get info on current job.
SYNC on entry;
Loop back (for potentially user has more than two.

Drawback: job DOES use a job slot,( but no other resources),

If that is cumbersome, then resubmit current job /after= & LOGOUT.

I hope this will be near enough what you want..

Proost.

Have one on me.

jpe

[Hoff]

Round-robin in the queue manager? Okfine...

AFAIK, there is no direct means to control nor to implement this scheduling without some custom coding; the OpenVMS queue manager doesn't consider the username or UIC as part of its scheduling and related processing.

You could use a stopped queue and some software.

Periodically scan the jobs submitted into the stopped queue, either on a timed basis, or upon receipt of an alarm for a queue write, and move the designated jobs over to the execution queue. DCL lexical functions or system services would work here.

Or you could use prolog processing in SYLOGIN that scans for other active jobs when the current job goes "hot", and self-requeues the current batch job based on your criteria.

Slightly fancier, roll your own server symbiont, and park the processing and the queue transfer within.

There are also some potential corner cases here, like what should happen when you have excess computes available. Should you force STAN to run tasks sequentially on the off-chance that OLLIE might need to submit something, even though the processor is otherwise idle and the capacity is available? Or when SNOWWHITE asks DOC to complete a series of tasks before she goes all pumpkin, and the round-robin scheduling is filled with the other six users.

I might also switch approaches (or mix approaches) and take a look at implementing the class scheduler here, too, and set it up to allow all of the the users to submit up to some number of processes, and allow the class scheduler to throttle activity accordingly. See SET PROCESS /CLASS and the sys$sched call and sys$getjpi[w] class stuff, as well as the SYSMAN> HELP CLASS details.

Here's some old (related) stuff, too:
http://mvb.saic.com/freeware/freewarev40/scheduler/

And FWIW, the better phrasing here is probably "not permitted to make changes", and not "can't make application changes". You and I know that we *could* make the necessary changes, even if we have to use an instruction-level patch to do it. :-)

Do ask the third-party for some assistance here, too. As they're certainly permitted to make these sorts of application changes.

Stephen Hoffman
HoffmanLabs LLC

[Jim_McKinney]

> roll your own server symbiont

Check out EXECSYMB found at some of the better FTP sites. I expect you could create a server queue through which all users would have to submit their jobs and that server could then funnel them into batch queues based upon your rules. You could control the batch queues by prohibiting direct submission - maybe using an ACL and then add/delete the UIC of the job owner from the queue's ACL while routing to the batch queue. Just a thought...

[EdgarZamora_1]

Thank you for the great ideas so far. It didn't even occur to me to look into putting some code in SYLOGIN. The "sync" won't work because that particular job would still be "executing" in the batch queue and taking a slot.

The idea of submitting to a "stopped" queue and having a job "process" the entries and requeue them has possibilities.

I was already looking into rolling out my own server symbiont but I'm not really sure I wanna go that far (same with class scheduling) just so that my phone stops ringing with users complaining about others hogging the queue. (Hmm maybe it's worth it).

Jim, thanks for the pointer to execsymb. Another system here is actually using that (not sure for what) but I'll take a look at that.

[John Gillings]

Hi Edgar,

Maybe instead of hard limiting your users to a specific number of jobs, you penalise them for each job over your defined "reasonable" limit. Increase the job limit on the queue, so that more users can get their jobs running, but reduce the priority of additional jobs submitted by the same user.

I've posted some (untested) code to scan for batch jobs running under the current username. As written it allows 2 jobs to run at normal priority, then decrements the priority by one for each additional job.

The benefit is a user CAN exploit all the job slots if they're free, but if there are multiple users, the first two jobs for each user get CPU priority.

$ ctx = ""
$ temp = F$CONTEXT ("PROCESS", ctx, "NODENAME", "*","EQL")
$ temp = F$CONTEXT ("PROCESS", ctx, "USERNAME", F$EDIT(F$GETJPI("","USERNAME"),"COLLAPSE"),"EQL")
$ temp = F$CONTEXT ("PROCESS", ctx, "MODE", "BATCH", "EQL")
$ prio=F$INTEGER(F$GETJPI("","PRIB"))
$ me=F$GETJPI("","PID")
$ newpri=prio+2
$ loop: pid=F$PID(ctx)
$ IF pid.NES."".AND.pid.NES.me
$ THEN
$ newpri=newpri-1
$ IF newpri.GT.0 THEN GOTO loop
$ ENDIF
$ IF newpri.LT.prio THEN SET PROCESS/PRIO='newpri'
$ IF F$TYPE(ctx) .eqs. "PROCESS_CONTEXT" THEN temp = F$CONTEXT ("PROCESS", ctx, "CANCEL")
$

(of course, a user could just reset their priority with SET PROCESS/PRIORITY - if that becomes an issue, a SMOP to write a privileged program to reduce the permanent base priority of a process, also safe to install privileged, as long as all it can do is permanently reduce process priority)

Note that a job limit of 6 on a modern Alpha seems a bit low! Most systems can happily run hundreds of simultaneous jobs.

Per-user batch queues, protected by ACL shouldn't be too much of a problem if you write some simple scripts for maintaining accounts. Whenever an account is created, create the queue, and delete it when the account is deleteed. Define SYS$BATCH in SYLOGIN according to username.

[EdgarZamora_1]

Hi John. Very interesting idea! Definitely going to look into and thank you for the code.

The job limit of 6 isn't that low since that's just one batch queue and there are a bunch of queues for different departments. I had just used "sys$batch" as an example.

[RBrown_1]

Sorry to wake an old thread, but I have to ask: What about /MAXJOBS in the UAF? According to HELP, this lumps interactive, batch, detached, and network together, but if you are limiting batch jobs per user, perhaps you are limiting the other modes as well.

[RBrown_1]

>> What about /MAXJOBS in the UAF?

Never mind, I should have tested it before I tried it.

When maxjobs is reached, subsequent jobs are deleted rather than left in the queue. :-(

[EdgarZamora_1]

Thanks for looking into it. Appreciate that. Yes, maxjobs won't do the trick.