AUTOSTART=* for INIT/QUEUE

[Jan van den Ende]

On HP Advocacy,

http://www.hpuseradvocacy.org/node/1992

Issue 1992 is open for voting.

Essentially it is about realising the title subject, because now you HAVE to specify the list of nodes by name.
The rest of the reasons is explained in Advocacy 1992.

Poits await those that report here that they did
" I Agree",
but you will have to wait till after my holiday.

Ski Heil!

Proost.

Have one on me.

jpe

[Duncan Morris]

Been there, done that.

Have a great holiday.

[Karl Rohwedder]

Sounds logical, I agreed.

regards Kalle

[Marc Van den Broeck]

Undoubtly useful so I agreed.

[Wim Van den Wyngaert]

I agree but would propose also an option in which you say "those with lockdirwt > 0". That would exclude quorum stations and cluster workstations in general.

Wim

[Ian Miller.]

been there, done that :-)

[Phillip Thayer]

I have wished for this for a while. This would be great if they could do it. Also, if they could have it add a node name to the list whenever a new node is added to a cluster.

Phil

[Volker Halle · Accepted solution]

Jan,

maybe I'm misunderstanding something, but:

$ INIT/QUE/BAT/AUTOSTART=(node1::,node2::) TEST

- accepts non-existing nodes
- the list of nodes can be extended without deleting the queue and it's entries

What is the problem ?

Do you really need /AUTOSTART_ON=* because you have so many nodes in the cluster and don't want to type the whole list ?

Volker.

[Bart Zorn_1]

I agree with your topic, if not with your greeting!

If you really have to recreate queues, ASSIGN/MERGE is your friend!

Regards,

Bart Zorn

[Jan van den Ende]

Thanks, all.

Volker:

Thanks for pointing out that
$ INIT/QUEUE /someparam does NOT loose pending jobs! That DOES take some of the pain away.
However, the basic issue remains: in a cluster with many (our case: 1000+ , expected to grow significantly in the near future) batch and print queues, it stiil is a major task to catalogue for each queue whether it is a print, a batch, or a generic queue, and whether it needs to run on any, node, or just a specific one.
(Wim, thanks for your suggestion, it exends the previous line to include "one of some nodes" )
Seeking out which queue needs which modification and re-specifying would be eliminated by the requested addition.

Bart:
on the greeting: it is the common Tyrolian salute to wish much fun with skying and everything around it (and for the locals probably also includes a wish for success in the ensuing business). If I hit any undesired sensitivity somehow, please accept my apologies.

Proost.

Have one on me.

jpe

[Ian Miller.]

hp have responded to
http://www.hpuseradvocacy.org/node/1992
asking for some more detail.

[Jan van den Ende]

Yes Ian,

I received their notification.

Interestingly, the y ask for some more info, but the voting has closed and I havo not yet found an easy way to respond.

Hopefully they watch this thread as well (could someone point the relevant Engeneers to this?).

Let me try to give MY reactions to the questions (anybody having other/more please chime in).

- the initial reason was mainly the print queues, for which the issue is the most relevant and (as far as I can judge) the issues raised are minor.

- for mixed architecture clusters, I have the feeling that those do require certain extra management effort. I tend to count among those the careful evaluation of which queue should be available for which architecture. I concede that those wil not benefit from the enhanced proposal.

- in case of intended node preference(s) for batch queues, the current mechanism should remain available (it should anyway: "do not break existing code".


But: IF a decent load balance scheduling mechanism could be implemented, of course that would be nice!
BTW: is there any such mechanism NOW in case of failover? The way queues behave in our cluster suggests otherwise.


hth.

Proost.

Have one on me.

jpe

[Ian Miller.]

I hope that responses here will be seen.

[Volker Halle]

Jan,

I believe the delete and re-create queue and jobs issue has been taken care of.

If you would really want /ON=*, then you as a system manager need to guarantee, that the identical environment is present on all your nodes in the cluster. I'm thinking of symbionts (e.g. DCPS installed on all systems ?) or environment for batch jobs to run (same applications available on all nodes, do your batch command procedures run on all architectures ?). This is not something the QUEUE_MANAGER can determine.

Volker.

[Jan van den Ende]

Volker,

OF COURSE it is the task of the system manager to ensure the environment is in agreement with the settings (and vice versa)!
But that is exactly the issue: in case of homogeneous clusters, when a node is added or replaced, A LOT of work has to be done to KEEP the cluster homogeneous, where /ON=*, or /AUTOSTART_ON=*, would take care of all those queues that SHOULD ensure homogeniety!

I have clearly asked to ADD this functionality, NOT to replace the old mechanisms!

Proost.

Have one on me.

jpe

[Volker Halle]

Jan,

so what about something as simple as this:

$ DEFINE/SYSTEM ALL_NODES nodea::,nodeb::,nodec::
$ INI/QUE/AUTOSTART_ON=(ALL_NODES) queue

Then just teach INI/QUE to try to translate the 'node' name parameter and accept a list of values. The system manager is responsible for setting up that logical and making sure, that the ENVIRONMENT is the same on all nodes.

Until this feature would be available, you could set up an ALL_NODES symbol on your cluster and use it in your queue startup procedure:

$ all_nodes:==nodea::,nodeb::,nodec::
$ init/que/autostart_on=('all_nodes') queue

Volker.

[Jan van den Ende]

Volker,

That second idea might be a decent workaround, combined with your previous info about submiied jobs staying in the queue.
I will do some testing with executing jobs in a queue being (re-) INITted.

Thnx

Proost.

Have one on me.

jpe