topic Re: MC/ServiceGuard and application start/stop scripts in Operating System - HP-UX

MC/ServiceGuard and application start/stop scripts

David Niska — Fri, 06 Dec 2002 22:01:55 GMT

I have an application running a progress database that has a series of start/stop scripts. These scripts perform the start/stop of the application processes and the progress database perfectly when ServiceGuard is NOT running. I have been able to cluster the two nodes this app. runs on, and get the application configured into a package, but when I run the application start/stop scripts, they do not work properly. In fact, the stop script hangs until the DBA manually shuts the database down. Then the stop script will continue to shut everything else down properly and unmount the filesystems. Has anyone ever seen anything like this before?

In summary, the scripts work fine when ServiceGuard is NOT running and don't work when it is. Same exact scripts in both instances.

Re: MC/ServiceGuard and application start/stop scripts

Sridhar Bhaskarla — Fri, 06 Dec 2002 22:08:57 GMT

Hi David,

You will find more details in /etc/cmcluster/your_package_name/your_package_name.ctl.log

The above file name is dependent on your package name.

When you are halting the cluster, make sure you do a tail -f of the file and see the progress. You will get good clues from there.

-Sri

Re: MC/ServiceGuard and application start/stop scripts

Ashwani Kashyap — Fri, 06 Dec 2002 22:12:45 GMT

What does your package control script log says. There should be some information there that can give a clue whats wrong .

THe log file should be in /etc/cmcluster//package_name.cntl.log .

Also look for any errors in syslog.log file .

Re: MC/ServiceGuard and application start/stop scripts

Jeff Schussele — Fri, 06 Dec 2002 22:18:18 GMT

Hi David,

Remember that you don't have a "real" environment when the MC/SG script runs the start/stop scripts. You need to make sure the scripts source whatever env vars you need to run them. Set up all the needed env vars in a file and source that first thing in the start/stop scripts.
I suspect that some var(s) are not being set when MC/SG runs the stop script.

Rgds,
Jeff

Re: MC/ServiceGuard and application start/stop scripts

David Niska — Mon, 09 Dec 2002 14:36:40 GMT

Here is some more information:

I am sourcing in the environment variables at the beginning of the start/stop script. In the package.cntl.log file I get a message that the shutdown is executing which translates to a proshut command for the progress database. That is where it hangs.

I don't get any helpful information from the log. Basically what's happening is that the progress database shutdown script will not shut the database down properly when ServiceGuard is running. These scripts work fine when MCSG is not running. I am currently looking into any environmental issues that could be causing this. Thanks,

Dave

Re: MC/ServiceGuard and application start/stop scripts

David Niska — Mon, 09 Dec 2002 16:27:05 GMT

In looking at the differences between the environments, I found that the shell parameter for MCSG is set to /usr/bin/sh and it is set to /usr/bin/sh without MCSG. Also, the PATH variable does not have all of the paths as when MCSG is not running.

Where does MCSG get it's environment variables?

Dave

Re: MC/ServiceGuard and application start/stop scripts

David Niska — Mon, 09 Dec 2002 16:29:39 GMT

Sorry, that would be /usr/bin/sh when MCSG is running and /sbin/sh when it is not.

Dave

Re: MC/ServiceGuard and application start/stop scripts

Oleg Zieaev_1 — Mon, 09 Dec 2002 16:36:28 GMT

Do you have designated user to manage the database?
If so, use this user to control your start and stop procedure.
In your case it will translate to line like this in package.cntl for stop user defined apps:

su - proguser -c ". .profile; proshut > /log/proshut.log 2>&1" &

This approach we use for oracle and it's working like a charm :)

Hope this helps,
0leg

Re: MC/ServiceGuard and application start/stop scripts

Oleg Zieaev_1 — Mon, 09 Dec 2002 16:39:49 GMT

One more important thing:
add wait right after that line.

'&' in this case is not needed.
This will wait for DB to be down before continue and umount lvols.

Hope this helps,
0leg

Re: MC/ServiceGuard and application start/stop scripts

David Niska — Mon, 09 Dec 2002 16:44:22 GMT

I already have the "su user -c shutdown script" in my start/stop script and the problem is that the start/stop script hangs and will not shutdown the database or kill the processes.

In order to get the package to shutdown, we need to kill off the stop script process and run the proshut manually.

Let me know if you have any other ideas. Do you know where MCSG gets it's environment variables?

Dave

Re: MC/ServiceGuard and application start/stop scripts

Oleg Zieaev_1 — Mon, 09 Dec 2002 18:56:22 GMT

David.

First: you need su - user -c
(su minus user)
to activate all environment variables.
Just to ensure that I also added . .profile; before proshut. If .profile you can set any env. variables your DB requires.

If you created package files from the template, you have all variables you need for MC/SG. Env variables required by application has to be initalized in your startup/shutdown script.

If you login as proguser, you can issue commands to shutdown your database, right?
So, I do not see any reasons for the line I mentioned earlier to fail to do the job.

WAIT is required, so script will not start umount and vgchage before DB is down.

Hope this helps,
0leg

Re: MC/ServiceGuard and application start/stop scripts

David Niska — Wed, 11 Dec 2002 14:25:07 GMT

This problem has been fixed! After several days of working with HP on what could possibly be happening, I talked with an engineer that said that I needed to load patch PHSS_26270 in order to remain on a supported version of MCSG. I loaded this patch last night and all of my problems went away. Everything works great now! Thanks for all of your help.