shutdown issue

[josutilities]

1) VCS stop scripts[K110vcs ] executes offline node in the cluster . /opt/VRTSvcs/bin/hastop -local -noautodisable > /dev/null 2>&1
2) After maestro Service Group sgotcpmaestro offline procedure did not complete within the expected time, maestro_clean script rans
***************************************************
$ cat maestro_clean
#! /usr/bin/sh
#
#
# Meastro start script for VCS.
# This script has multiple links, one for each instance of maestro.
# Instance that will be executed will depend upon the link with which
# the script is called.
#
# Created 11/18/03

export MAESTROUSER=maestro

PIDS=`ps -ef | grep $MAESTROUSER | grep -v -E 'grep|_clean' \
| awk "BEGIN { ORS = \" \"} { print \\$2 }"`

kill -9 $PIDS
$

3) Shutdown gets killed automatically [nohup /sbin/shutdown -ry 0 >>/usr/local/logs/shutdown.log 2>&1 & ] .


Any body can help me why Shutdown gets killed automatically



Thanks In Advance

[Ralph Grothe]

I only have a VCS running on Solaris nodes to compare.
But you may be relieved to note that in our vcs init scripts the cluster services are halted almost identically.
This is the excerpt from the stop case/esac block:

'stop')
$HASTOP -local -evacuate -noautodisable > /dev/null 2>&1
$CMDSERVER -stop
;;

Ours only seems to deviate in that it uses the -evacuate switch, which according to man hastop takes care to bring up the so far locally active service groups on the failover node(s).
Strangely, the -noautodisable doesn't get mention in the manpage at all.

Since we don't use an application/service group called MAESTRO I cannot help you there.
But the SIGKILL you send in the Maestro script to a parsed PID from the proc table (hopefully the right one) looks pretty daring to me.

I don't quite get your 3rd question.
Does it refer to a manually executed shutdown?

[josutilities]

While scheduled reboot of one of my cluster node shutdown gets hung.

steps 1-4 shows the sequence

1) AWSBIS307I Starting /opt/maestro/jobmanrc /usr/local/bin/shutdown.sh

Content of the usr/local/bin/shutdown.sh
cat /usr/local/bin/shutdown.sh
#!/bin/ksh
cd /
nohup /sbin/shutdown -ry 0 >>/usr/local/logs/shutdown.log 2>&1 &
exit 0

2) VCS stop scripts[K110vcs ] executes offline node in the cluster .
/opt/VRTSvcs/bin/hastop -local -noautodisable > /dev/null 2>&1

3) After maestro Service Group sgotcpmaestro offline procedure did not complete within the expected time, maestro_clean clean script for the app. resource script rans
***************************************************
$ cat maestro_clean
#! /usr/bin/sh
#
#
# Meastro start script for VCS.
# This script has multiple links, one for each instance of maestro.
# Instance that will be executed will depend upon the link with which
# the script is called.
#
# Created 11/18/03
export MAESTROUSER=maestro
PIDS=`ps -ef | grep $MAESTROUSER | grep -v -E 'grep|_clean' \
| awk "BEGIN { ORS = \" \"} { print \\$2 }"`
kill -9 $PIDS
$

5) Shutdown gets killed automatically [nohup /sbin/shutdown -ry 0 >>/usr/local/logs/shutdown.log 2>&1 & ] .

I did a ps -ef | grep -i shut ,
i couldnt find the /sbin/shutdown -ry 0 executed in nohup mode.

[Ralph Grothe]

Hm, pretty confusing.
Does this mean that you can't shutdown the node at all,
or is only your shutdown.sh wrapper script failing?
(I guess the latter)
I don't no if it would change a thing,
since I still can't overlook the Maestro and VCS script interaction (I guess Maestro is a VCS service group (SG) that needs to be failed over when you reboot the node)?
Alternatively to nohup you could try running shutdown as a mere batch job like

echo /sbin/shutdown -ry 0 | batch

You may add your redirections of stdout and stderr, but what is the purpose?
The /etc/rc.log isn't sufficient?