script behaves different when run under rc

[C B Smith]

Hello all,

I have a script which is running some high availability software. When I run this script from a command line after logging-in the script works as expected. When I have rc run this script at startup by putting a link in the /etc/rc5.d/ (etc.) tree the script exits early.

First a little about my script. it is fairly straightforward. the core of is similar to below...

# a bunch of environment stuff
(while ((1)) ; do
myspecialprogram -options
logger myspecialprogram exited!
echo myspecialprogram exited! >> testfile
kill myspecialprogram_pid
#implicit restart
done ) &

So this script is intended to keep this program alive forever. even if the program seg-faults or does some other nasty stuff internally.

What I have found is that when run at startup when myspecialprogram exits the whole script exits. The program starts fine, but if it exits, (like if i do a kill -SEGV pid -- note pid not ppid) the whole process tree vanishes. It does not even log anything to the syslog or my test file, it is just gone. When I run it later after logging in as a root or a user it works just fine. When the program exits it is restarted as expected.

Now I have definitely seen this work just fine under solaris and tru64, now I am trying to run under RH9. This is using the bash shell under RH, under the others it was sh I believe.

Before jumping to any environment related conclusions, everything that myspecialprogram needs environment wise it has setup in this script. It can be run from just a straight root login. Environment really should not effect the while anyway, unless I am missing something.

i have also removed the kill incase that killed the parent process also, but that did not affect the problem.

Any ideas? I am not very knowledgable on how starting up from within RC is different than from a prompt. Could there be any environment differences that would effect bash itself?

thanks,
chris

[Mark Grant]

I must say, bash does have a few oddities that MIGHT be considered OK but I personally don't agree that they are.

Personally, in instances like this, I would simplify. I am not particularly fond of your (while ((1)); do
blah
blah
) &

I would firstly, use "while true" and secondly, try putting that whole loop in a separate script and run that from the rc script.

[Olivier Drouin]

rc scripts are not meant to use loops in them.

Use init to keep your process alive.

$ man init

[Steven E. Protter]

rc scripts start daemons.

They are not meant to be daemons.

Your methdology needs to change.

Have the rc script start your daemon, write it anyway you want.

SEP

[Andrej Vavro]

Chris,

I had similar experience with setting tape block size after each boot. It is unable to use uma (omniback tape manipulating tool) from rc, cron, inittab. But it works very well from bash. Even the same behavior on HP-UX.
There is no problem with mtx - linux tape manipulating tool (SCSI robots).

Andrej

[Jeroen Peereboom]

Chris,

I agree with SEP: you must start a deamon within your script, so you need to make some changes. Maybe using 'nohup' will help.

Also consider the suggestion in a previous reply to use init to keep your script alive (add an entry in inittab with a respawn option).

JP.

[C B Smith]

Well, I appreciate your comments. this is giving me a great excuse to clean up this code.

I tried pulling my loop out into a separate script with no luck. having a separate "looponly" script made no difference.

#!/bin/sh
#args broken out for clarity
p=$1
shift
ARGS=$@
(while true ; do
echo $p $ARGS
$p $ARGS
sleep 2
logger Restarting $p
done
) &

unless im misunderstanding the defn of daemon that is what this script is. a very simple daemon which keeps my program alive. it just uses bash instead of something else. note the & runs the (...) out of process so the rc script itself continues on as all other rc scripts do (both in this version and the previous). it is just the "mini" script in the () that persists as its own process.

ive considered using init, but i consider that a last resort since it means i will be unable to stop my program without altering the runlevel. much nicer to just a have a killable process keeping it alive. But for some reason this doesnt work under using a bash script under rc...

as for the earlier loop comment, there are lots of loops in the exisitng rc scripts... see /etc/rc.d/init.d/functions

Any ideas about what is different here? Why does bash behave differently under rc?

Is there a way to get rc to rerun after boot? Will it re-run if I run init() to change to another run level?

chris

[Mark Grant]

I can send you an alternative script if you like. It's a bit over-kill but amongst a multitude of other things, it does monitor applications, restart them if you want and alerts too. Comes with a nasty little GUI admin script too. Here is a tar ball of the thing but please read the README and the WARNING.

If you have any trouble with it e-mail me. We find it useful here.

[C B Smith]

ah ha! found it. i was not redirecting stdout/err so my programs were getting a sigpipe when they were restarting, terminating my script.

I can't entirely explain why, but it almost makes some sense to me... if anyone can explain it please do. I suspect that when starting up bash is in a mode that redirects stdout/err then when the program restarts later that special mode has ended so it no longer has a "pipe" to its stdout. ?? if anyone can explain it please do. :)

adding a "> /dev/null 2>&1" to my while loop makes it better.

thanks for the help.

chris