Service Guard Failover Config

[Mike_316]

Hey Gang,

Different type of question, which I did not see a clear answer to in the SG documentation.

Is it possible to configure service guard to, upon the failure of the primary node, verify if there are specific PROCESSES running on the secondary node (NOT to see if the node is simply "up", but check for specific processes) and use the process information of the secondary node to determine if it continues on the secondary node, or moves onto the tertiary node?

Thanks!

Mike

[John Poff · Accepted solution]

Hi Mike,

You can code that kind of logic in your customer_defined_run_cmds function in your package control script, but with more than two nodes it could get interesting.

JP

[Mike_316]

Thanks John! You wouldn't happen to have a link off the top of your head which addresses that specific type of configuration, would you?

[James R. Ferguson]

Hi Mike:

In one sense, in effect, you are asking for a multiple node failover. Consider the case where the primary node panics for reason of a hardware failure. The second(ary) node acquires the package as designed, but then, depending on what else its running, must divest itself of the package.

One way (obviously) is for the second node to panic (assuming the package can move again).

I suppose that you could interrogate your environment and if you found a particular set of conditions you (a piece of code you wrote) could halt the package and move it to the tertiary node.

Regards!

...JRF...

[John Poff]

Mike,

Try the 'search' option on the forum. I know I've seen similar discussions here about how to code things in those functions. I'm sure you'll be able to find some threads there.

JP

[Ashwani Kashyap]

Hmm , interesting question .

I think one way of doing it could be like this . IF your processes on secondary node could be started from a package/packages , then SG package on the primary node could be configured to failover to tertiary node using the MIN_PACKAGE_NODE variable in the package conf file .

Another way , I think is to write a script/function and put it in the primary nodes package control file where in checks the following before VG activation and checking and mounting file systems :

a . If running on primary host , then start normally .

b. If running on secondary node , then look for PROCESSES , if processes not found then startup normally , else do a cmrunpkg on the tertiary node .

Or you can build any logic that suits your purpose .


I hope others provide a better solution .

[Martin Johnson]

We set up a 3 node cluster where when node1 failed, the packages would fail over to node2, but before the packages would start, it would cause the packages on node2 to fail over to node3. (The packages on node3 would be stopped to make room for the node2 packages.)

It took some time to get everything to work correctly. Unfortunately, somone had the brilliant idea to fail over only some packages from node1 to node2 to make data transfer between applications controlled by the packages easier. This wreaked havoc. Eventually, we set things up so node1 packages can automatically fail over to node2, but node2 was sized to handle both sets of packages. The fail over to node3 is now a manual task.

Be careful, test thouroughly! Remember KISS (Keep It Simple, Stupid)

HTH
Marty