1771005
Members
2611
Online
109003
Solutions
SOLVED
I have to say that the manuals, which are pretty good for most of Serviceguard, are pretty poor when it comes to describing how to actually set up an application within a package.
The thing to remember is that for the package component of Serviceguard, it's all scripts so you can pretty much customise it to your hearts content. The other important point is that YOU can change it! (not like Sun Cluster where Sun PS have to get involved - I once remember a Sun Cluster which took 2 weeks to get Sun PS to change an IP address!).
Anyway, I'm assuming you've had a look at a skeleton package control file as created by running:
cmmakepkg ├в s /etc/cmcluster/pkg1/pkg1.cntl
...or whatever.
The two important parts in this are the functions customer_defined_run_cmds() and customer_defined_halt_cmds(). This is where you start and stop your app, as you seem to have already worked out... straightforward so far.
What about monitoring the status of an application? Well you were already pretty close to the answer to this when asking about the difference between packages and services. A package is a group of data(logical volumes/filesystems), processes and IP address(es) that make up an application. A service is part of a package in that it is started and stopped by the package - but Serviceguard can take special actions based on what happens to services.
What most people use services for is exactly what you want - monitoring applications. So how do we set one up?
First of all a service needs to be defined in the package configuration file AND the package control script. Lets assume we are creating a simple process monitor for your application (does the database process exist in the process table - i.e. 'ps -ef | grep ...' - you get the idea!) First of all we need some lines in the pkg1.conf file:
SERVICE_NAME db-mon
SERVICE_FAIL_FAST_ENABLED NO
SERVICE_HALT_TIMEOUT 300
That's a name for our service, how long it should be given to stop on a SIGTERM (kill -15), and whetjer failure of the service should immediately cause this system to reboot (often referred to as transfer-of-control or TOC in HPUX speak).
Now we need to add some lines to our control script, you'll find a commented out example already in there, but we will add something like this:
SERVICE_NAME[0]=db-mon
SERVICE_CMD[0]="/etc/cmcluster/pkg1/db-mon.sh"
SERVICE_RESTART[0]="-r 2"
So thats the same name as from the conf file, the name of a script which actually contains your monitor commands, and an indication of what to do if the script exits - the '-r 2' means attempt to restart the monitor twice before issuing a failover alternatively '-R' means, just keep restarting the service infinitel
I am an HPE Employee
The thing to remember is that for the package component of Serviceguard, it's all scripts so you can pretty much customise it to your hearts content. The other important point is that YOU can change it! (not like Sun Cluster where Sun PS have to get involved - I once remember a Sun Cluster which took 2 weeks to get Sun PS to change an IP address!).
Anyway, I'm assuming you've had a look at a skeleton package control file as created by running:
cmmakepkg ├в s /etc/cmcluster/pkg1/pkg1.cntl
...or whatever.
The two important parts in this are the functions customer_defined_run_cmds() and customer_defined_halt_cmds(). This is where you start and stop your app, as you seem to have already worked out... straightforward so far.
What about monitoring the status of an application? Well you were already pretty close to the answer to this when asking about the difference between packages and services. A package is a group of data(logical volumes/filesystems), processes and IP address(es) that make up an application. A service is part of a package in that it is started and stopped by the package - but Serviceguard can take special actions based on what happens to services.
What most people use services for is exactly what you want - monitoring applications. So how do we set one up?
First of all a service needs to be defined in the package configuration file AND the package control script. Lets assume we are creating a simple process monitor for your application (does the database process exist in the process table - i.e. 'ps -ef | grep ...' - you get the idea!) First of all we need some lines in the pkg1.conf file:
SERVICE_NAME db-mon
SERVICE_FAIL_FAST_ENABLED NO
SERVICE_HALT_TIMEOUT 300
That's a name for our service, how long it should be given to stop on a SIGTERM (kill -15), and whetjer failure of the service should immediately cause this system to reboot (often referred to as transfer-of-control or TOC in HPUX speak).
Now we need to add some lines to our control script, you'll find a commented out example already in there, but we will add something like this:
SERVICE_NAME[0]=db-mon
SERVICE_CMD[0]="/etc/cmcluster/pkg1/db-mon.sh"
SERVICE_RESTART[0]="-r 2"
So thats the same name as from the conf file, the name of a script which actually contains your monitor commands, and an indication of what to do if the script exits - the '-r 2' means attempt to restart the monitor twice before issuing a failover alternatively '-R' means, just keep restarting the service infinitel
I am an HPE Employee
Now the hard part! you need to write a monitor script that does something useful! The typical bones of a monitor script would do something like this:
while true
do
Is process active? If so continue
if not attempt to restart, and then exit
done
Of course they can get a lot more complex than that if you want to monitor more things apart from the fact that a bunch of processes are running - I've written monitor scripts which attempt to update critical tablespaces in an oracle database, and time out if not updated in 30 seconds. I've attached a generic script I sometimes use as a starting point for this sort of thing.
So what happens is, your set all this up, and then if your software fails the script detects this, restarts the software and then exits - when serviceguard detects that the service has exits, it will restart the script the defined number of times before initiating a failover. So the upshot of all this is pretty much the same as with the Sun Cluster option (only in my opinion a lot more configurable!)
That said, I can't emphasise enough how important it is to read the Serviceguard manual, and attend a course - it amazes me that people will spend such huge quantities of money on creating a serviceguard environment and then not train their people to run it - A more illustrious member of the forums than me made an analogy recently between sysadmins and pilots - you wouldn't want someone with no training piloting your $50M jumbo jet would you?
Anyway this has probably raised as many questions as it has answered, but hopefully its been usefull.
HTH
Duncan
I am an HPE Employee
wierd... I had to submit that message in two parts - kept getting a message 'error while submitting reply' not sure why...
I am an HPE Employee
I am an HPE Employee