Hpux to Sco script getting stuck on startup..

[someone_4]

Hey everyone .
I am working on a script sitting on Hpux that will stop and start a procces on a many Sco Boxes. What is happening is that we have over 19 IVRs (Interactive Voice Response) that run on Sco. But everyone in a while the process needs to be stoped and started on each box. And for a while now we would have to telenet into each box and stop and start. But I am working on a script that will go in and stop and start everything from one box. I have tried it 2 ways. First and rcmd command from a sco box to the other sco boxes. *comments
#rcmd sco hosts -l ./stopivr *works
#rcmd sco hosts -l ./startivr * hangs
This works when I try to stop the process but when I try to start it then it gets stuck. And I would have to go and kill -9 the pid of the rcmd process. Then I did a remsh from an Hpux box and once again I can stop it fine but when I try to start it then it gets hung. * comments
#remsh scohost -l root ./stopivr *works
#remsh scohost -l root ./stopivr *hangs
The weird thing is that if I go check the box it gets hung on ( both from Hpux and Sco) The process is stuck yes .. but the remote process that I was try to start is started. But only on the first box the other ones are not. I have even tired
#remsh sco hosts -l ./startivr &
This gives me the command promt back but it still gets stuck. Anyone have any ideas?
Thanks in advance for your help

Richard

[someone_4]

Opps.
After reading
Theese two lines
#remsh scohost -l root ./stopivr *works
#remsh scohost -l root ./stopivr *hangs
should be
#remsh scohost -l root ./stopivr *works
#remsh scohost -l root ./startivr *hangs

Cuss I can stop it just not start it.
And I even tried to add the full path
remsh scho host -l root "PATH=/usr/bin:/usr/sbin;./startivr"

[John Poff]

Hello,

Does the startivr script require any input? Does it have any standard output?

I worked with several hundred SCO Unix boxes for over three years. This problem seems vaguely familiar. I'll dig around a little and see what I can remember. Of course, the old brain isn't what it used to be!

JP

[someone_4]

Yes it does have output but I have redirected it using. exec >&1 >> /tmp/output
And no it does not require any input.

Richard

[Patrick Wallek]

What about doing something like:

remsh scohost -l root "nohup startivr"

[someone_4]

I tried that and it still gets stuck on the host I am executing the script from.
And it is always the startivr that gets stuck.

[John Poff]

Richard,

Can you try the 'remsh' command with the -n option? This option redirects standard input to remsh from /dev/null. Please try this and let us know what happens:

#remsh scohost -l root -n ./startivr

JP

[someone_4]

#remsh scohost -l root -n ./startivr
still sticks ..

Richard

[John Poff]

Richard,

Can you post the startivr script please? I'd like to see what it is doing.

Thanks.

JP

[someone_4]

Here are the 2 scripts. Now this is not the script that does the remsh. This is the actull script that calls the ivr to stop and start.
On the remote host.



Here is the stop script

PIDS=`ps -ef | grep /usr/bin/ivr | grep -v grep | awk '{print $2}'`
if [ -n "$PIDS" ]
then
kill $PIDS
fi

PIDS=`ps -ef | grep /usr/bin/comcen | grep -v grep | awk '{print $2}'`
if [ -n "$PIDS" ]
then
kill $PIDS
fi

# do it again for those diehard respawning processes.

PIDS=`ps -ef | grep /usr/bin/ivr | grep -v grep | awk '{print $2}'`
if [ -n "$PIDS" ]
then
kill $PIDS
fi

PIDS=`ps -ef | grep /usr/bin/comcen | grep -v grep | awk '{print $2}'`
if [ -n "$PIDS" ]
then
kill $PIDS
fi
---------
here is the start script ..

echo "removing logs..."
cd /var/proc/log
rm *.log
df -tvk >/dev/null
/usr/bin/comcen
sleep 1
/usr/bin/ivr

[John Poff]

Richard,

Do you know where in the startivr script that things are hanging? Can you put an echo statement before each line and send the standard output and standard error to a file? If you can figure out which command it is hanging on that would be a big clue.

Also, I'm curious about the 'df -tvk' command. I understand what it does, but the output gets sent to /dev/null. Why is that command in the start script?

Can you give that a try? I'm getting curiouser and curiouser!

JP

[someone_4]

I ran the commands for the startivr script 1 by 1.
And I am getting stuck on
/usr/bin/comcen
and
/usr/bin/ivr
where do I put the echo to see the output ?

[John Poff]

Ok. I wonder if you need to start comcen and ivr through nohup. Here is what I would suggest to try (with the echo statements before each line):


echo "removing logs..."
cd /var/proc/log
rm *.log
echo "running df"
df -tvk >/dev/null
echo "starting comcen"
nohup /usr/bin/comcen &
sleep 1
echo "starting ivr"
nohup /usr/bin/ivr &


JP

[Patrick Wallek]

I think using nohup and starting the commands is a good idea. I think you will see your script work fine then.

Now the problem is making the change to all the scripts on your SCO systems! :)

[someone_4]

John do I edit the start ivr script with the nuhub and & or do I do this from the remsh script?

[John Poff]

Richard,

Edit the startivr script on the SCO box and put the nohup commands in there.

JP

[someone_4]

here is the output I get from the re written script. It seems to get stuck after sending out to nphup.out but if i hit enter my command line comes back.

removing logs ..
running df
starting comcen
Sending output to nohup.out
starting ivr
# Sending output to nohup.out

[someone_4]

It still gets stuck from the remsh command too.
Any more ideas?

Richard

[Shannon Petry]

If it were me....I would make sure to to this....excecute the command from a shell, with absolute paths. You have some debugging info in the script now, so should be able to see that it's not erroring on the script, but the launch...It should be a shell problem with the default shell.
remsh sco '/bin/sh /startivr'

Do not use a -n, because I have seen this hang certain systems. ESP BSD systems.

Check the paths above, and see if it works with them corrected. if /bin/sh is a link, use the real path, and not the link...same with startivr.

Regards,
Shannon

[John Poff]

Richard,

Just to eliminate another variable, I would try running the command between two of the SCO boxes. If it fails there, it is probably a problem with either the comcen or ivr programs. If it works, it must be a problem with the remote command machinery between HP and SCO. Either way, it should narrow down things a bit.

Also, when you run the script from a telnet session to the SCO box, do a 'ps -f' afterwards. I'd be curious to know what that shows.

I think I have another way to get it done, but we'll save that for a last resort.

This one is getting interesting.

JP

[someone_4]

good morning ..
Fist of all I want to thank you all for your help on this issue.
Let me start of telling everyone how I test this. First of all I have 2 telnet sessions open. 1 in the HPUX box that I am doing the remsh from and the second one where I am in the SCO box to see if the process stopped and started. Well now to what I discovered today.
I was doing what shannon suggested.
And in the SCO box the proccess were started and in the HPUX box they still got stuck and didnt give me back a command promt.
But what I did in the SCO box was a stopivr.
And that unlocked the script on the HPUX box and a command promt came back. I thought that was weird.
John , you suggest for me to do it from the SCO box to another SCO box. Is there any specific way for me to try this? Becuase just regular rcmd scobox2 -l root 'startivr'
gets stuck too. And when I do a ps -ef|grep rcmd I see the rcmd procces there.

Thanks
Richard

[Patrick Wallek]

Richard,

Can you post your new startivr script please.

When you added the nohup did you also add an & to the end of the command so that it would run in the background?

[someone_4]

Here it is .. it is in /usr/bin like the reg.
startivr
# more startivr2
echo "removing logs .."
cd /var/proc/log
rm *.log
echo "running df"
df -tvk >/dev/null
echo "starting comcen"
nohup /usr/bin/comcen &
sleep 1
echo "starting ivr"
nohup /usr/bin/ivr &

[John Poff]

Good morning Richard!

You just gave us more good information. Since the remote command is hanging up when you try it from an HP box and also from a SCO box, it sounds like the problem is with the 'startivr' program on the SCO box. I'm not a wizard with the remote command machinery (my Stevens networking books are at home-darn!), but it seems that the remsh/rcmd program is waiting for the program it executes (startivr) to finish up and return an exit code. It must do this from a regular telnet session, but something is causing it not to when it is run remotely

Ok. Looking at the remsh man page, it says, "..remsh exits when the sockets associated with stdout and stderr of the remote command are closed." Could it be that the 'startivr' program is sloppy when it comes to closing all the files and sockets it might have open?

Also, the man page for remsh says "The following command line causes remsh to return immediately without waiting for the remote command to complete:

remsh otherhost -n "command 1>&- 2>&- &"

So you could try:

remsh scohost -l root "./startivr 1>&- 2>&- &"

JP

[someone_4]

that didnt work. Same thing
is there something else I can do in sco or hpux besides a remsh or a rcmd ?