Cron in ServiceGuard

[Dan Hoyt]

Does anyone know a way to share cron between two systems in a two node Active/Passive ServiceGuard environment?

I can't really create a seperate /var/spool/cron/crontabs filesystem and fail it over because I still need standard root cron jobs to run on the passive system.

I tried links but soon found out those don't work because when you su - to that user and run "crontab -e" and make changes it over writes that link.

Any Idea's how I can accomplish this?

[Dan Hoyt]

My biggest concern is users (not knowing it's and HA environment) making changes to thier users crontab only to lose those changes when the box failsover. This happened the other day and I just looking for a posible solution to help resolve this issue.

[Alan Meyer_4]

create a data file(s) in a relocatable filesystem which contains the crontab entries needed for the package.

In your package control script, place coding that greps for the crontab entries in erspective crontabs and copies them out as needed.

In the package shutdown scripts, place coding that greps teh unnecessary entries out of the respective crontabs.

[Mel Burslan · Accepted solution]

in your crontab, create an entry which runs as frequently as you wish, as

============================
rcp /var/spool/cron/crontabs/root $passivenode:/var/spool/cron/crontabs/root; remsh -h $passivenode /usr/bin/crontab -n /var/spool/cron/crontabs/root
============================

(NOTE:in between two lines with multiple "=" signs goes as a single line into crontab)

in your package control file, you will need to make a determination of which node is the activenode and which is passivenode and record it in a file and use this file to determine the $passivenode variable here. The determination logic can go into the user defined commands section of package control file. Like :

#this example assumes a 2 node cluster
my_cluster_ascii_file=/etc/cmcluster/sapTESTdbci
node1=`grep NODE_NAME $my_cluster_ascii_file|head -1|awk {'print $2'}`
node2=``grep NODE_NAME $my_cluster_ascii_file|tail -1|awk {'print $2'}`
activenode=`hostname`
if [ "$activenode" = "$node1" ]
then
passivenode=$node2
else
passivenode=$node1
fi
echo $passivenode > /var/adm/CLUSTER_passive_NODE
echo $activenode > /var/adm/CLUSTER_active_NODE
#end of code snippet


Hope this gives you an idea. I know it has holes in it as I do not know your environment but it is a guideline not a solution off the bat.

[Mark Grant]

This is a fun problem that hasn't been properly addressed by HP in my view. Copying crontabs around all the nodes can work as long as it doesn't matter if your cron jobs run on the wrong machines and that you find some way of getting cron to re-read its crontab files (kill -1 will do it). Also, if packages switch, you still need to copy crontabs around from the new node. It all can get a bit nasty.

For me, the best way would be if cron followed symbolic links and you could link the crontabs to files on relocatable filesystems. Sadly though, it doesn't.

The only solution I have come up with is similar to your first reply here. I keep crontab files relating to an application in the application's filesystem. The startup scripts for the package copy the crontab files out and signal cron to re-read the files. These need to be copied back regularly, so theres a cron job to do that. It's a kludge in my view.

[Stephen Doud]

cron and password synchronization are two system administration areas that are not directly addressed by the Serviceguard software lab.

On occasion (a high number of requests) the HP Response Center attempts to bridge this gap by writing Engineering Notes.

Implementing additional root-owned cron jobs based on package existence on a node has been addressed in a document (UXSGKBQA00000066 - "Updating cron when ServiceGuard packages start on the node").

However, insuring user-owned cron and 'at' jobs are transferred to adoptive nodes at package failover has not been addressed by a document to my knowledge.
Obviously, transferring cron jobs to the adoptive node must be done before node failure. It would be best if the joblist was always accessible to the active node regardless of which node is in 'active' state. Placing the cron jobs on a failover file system is a good method because the cron jobs will only be executed if they are found on a users' file system.

Another method is the create a wrapper script which captures cron requests and copies them to the other server, but complexity is added if there is a dependency on 'active' status of the node.

[Doug O'Leary]

Hey;

The way I generally do this and tell users to do it is to maintain the same cron script on both systems and include logic similar to the following:

pkg='dbE1P'
Host=$(hostname)
P_host=$(/usr/sbin/cmviewcl -p ${pkg} | tail -1 | awk '{print $NF}')
[[ "${Host}" != "${P_host}" ]] && exit

That will exit out of the script if the package isn't running on the current host.

HTH;

Doug

[Dan Hoyt]

Thanks everyone for your replys. I've taken a little of all your ideas and put a process in place.