Serviceguard
cancel
Showing results for 
Search instead for 
Did you mean: 

packages can not failover when process down

 
mk707
Occasional Contributor

packages can not failover when process down

I created 2 nodes cluster as failover type, and 4 packages, but my packages could not failover.
Could you please advice this issue?

node1 and node2 has AUTO_RUN setting.
all packages set aut_run yes, failover_policy configured_node.
pkg1 provides VIP with and monitored_subnet X.Y.Z.0.
pkg2 provides shared disk.
pkg3 provides ftp server as external_script with dependent on pkg1 up state and pkg2 up state.
external script invokes vsftpd using /etc/init.d/vsftpd start/stop.
pkg4 monitors vsftpd process and exit with status=1 when missing process.

I want to failover all packages when vsftpd down, but it was not.
When I killed vsftpd process, nothing happened. Only pkg4 move to down status, but the other packages were still running on node1.

I need another scripts or another settings to move all packages to standby node?

Best regards,
Keiko

6 REPLIES 6
Bill Hassell
Honored Contributor

Re: packages can not failover when process down

The logic used to monitor the availability of vsftpd may need some rework. The program vsftpd (https://security.appspot.com/vsftpd.html) seems to be a typical daemon which usually is scheduled through inittab with a respawn action in case the program disappears. This would be the preferred design to ensure that the daemon is always running. 

If pkg4 detected the loss of vsftpd, then the other 3 packages need to be shutdown. So the settings you are using for all the packages needs some additional work.

 



Bill Hassell, sysadmin
mk707
Occasional Contributor

Re: packages can not failover when process down

Hi Bill, thank you for your advice.

I know Serviguguard package needs to "always running". At first, I defined /etc/init.d/vfstd as service_cmd in pkg3, but I got fail because /etc/init.d/vsftpd script immediately exited. (SG manual said this is bad definition.)
So, the second time, I temporally added sleep 1d at last of /etc/init./vsftpd and I put /etc/init.d/vsftpd into start function in external script. But this way did not success, because the package status was startting not up state.
The third time, I changed start function from my tempolally vsftpd to original /etc/init.d/vfstpd. Then pkg3 went to run state.
Now I recognized daemon program such as vsftpd must execute as external script.

My pkg4 watches vsftpd process. pkg4's function is as follows.

while true
do
echo `date`
# string=`ps -e | grep "vsftpd.conf"|grep -v grep` > /dev/null
string=`ps -aef | grep "vsftpd.conf"|grep -v grep`
if [ $? -eq 1 ]; then
exit 1
else
pid=`echo ${string} | awk '{print $1}'`
if [[ $pid = "" ]]; then
return 1
fi
fi
sleep 60s
done

Bill Hassell
Honored Contributor

Re: packages can not failover when process down

A few hints on your script:

The ps command can find a process by name (-C option). No need for the double grep.
The UNIX95=1 construct turns on extra options just for ps (see: man ps)

Not sure about vsftpd.conf. In the script below, I assume that the process you want to monitor is vsftpd.

while true
do
     UNIX95=1 ps -C vsftpd > /dev/null
     if [[ $? -ne 0 ]]
     then
        return 1
     fi
     sleep 60
done

 



Bill Hassell, sysadmin
Patrick Wallek
Honored Contributor

Re: packages can not failover when process down

Why do you have all of the different resources in different packages?  This is not how a SG package is typically designed.

 

For a single service like this, you typically have your network, shared disk, running process, and process monitor all in a single SG package.  Then if the process goes down the entire package, encompassing all needed resources, fails over to the other node.

 

 

mk707
Occasional Contributor

Re: packages can not failover when process down

Hi Patrick

Thank you for your advice, 

I understand SG design by your advice. I will try it.

Keiko

Highlighted
mk707
Occasional Contributor

Re: packages can not failover when process down

Hi Bill

I could pickup ftp process  perfectly. Thank you.