Service Guard

[Richard Barcellano_1]

Hi,

When the A package is starting in the OSS, it requires so much time until the 1, 2, and 3 package is timing out and disabling automatically. The packages are being enabled manually several times before the db package is complete.

Can someone please help me solve the problem so that customer will not anymore manually enable the oss packages in order to start the cluster. Is there a parameter we can change in the service guard cluster configuration?

Thanks in advance.

[Jeff Schussele]

Hi Richard,

There are several ways to approach this:

1) Put a wait loop in the start of the 1,2 & 3 pkgs & force them to sleep for a while - long enough for the DB to come up - before they actually run their startup commands.

2) Put a "lock" file out there that will only be removed when the DB is fully up & then set up the 1,2 & 3 pkgs to not start until the lock file is gone.

3) I assume you have some sort of monitor for the 1,2 & 3 pkgs - you could simply disable it & set the retry count on them very high or even unlimited.

I'm sure others will have even more suggestions. I think the dependency-based idea the best by using a lock file.

Rgds,
Jeff

[Sridhar Bhaskarla]

Hi Richard,

This isn't the default serviceguard behaviour. So, there isn't a simple switch in the serviceguard configuration to make it work.

Looks like your serviceguard scripts were written in such a way if the db pacakge is not up, the packages 1 and 3 won't come up (or timeout). I would suggest the following in addition to Jeff's suggestions assuming that you need to have packages 1 and 3 are dependent on the database.

1. Make AUTO_RUN (previously PKG_SWITCHING_ENABLED) to NO for both packages 1 and 3.

2. Have your db package start these packages after it itself comes up completely.

3. Have your db package monitor these packages and start them if found down.

You will need to do a bit of scripting as there are couple of things you will need to take care of. For ex., bringing up these packages on their primary nodes or alternate nodes if the primaries are not healthy etc.

-Sri

[A. Clay Stephenson]

I don't like the idea of lock files because where do you put them? I wouldn't want to make them NFS files because that opens another point of dependency for the packages. I wrote an article for the April '03 Issue of SysAdmin that addresses a problem very similar to this.


http://www.samag.com/documents/s=7898/sam0304a/0304a.htm

The idea is that rather than using lock files, you use Perl sockets to create a set of semaphores visible across multiple platforms. You can download the server.pl and client.pl pieces for sysadmin. I would start the semaphore server along with package A. The semaphore server should use Package A's IP address. You other package startup scripts can loop until 1) the semaphore is accessible and 2) the semaphore becomes the desired value -- as set by Package A. The beaty of this approach is that it will work no matter which host is running which package.

The good news is that all the hard stuff is done inside the Perl "black boxes". All you have to add is a little shell scripting.