HP server to tell Linux server to shutdown on power failure

[Marty Metras]

We have 2 HP-9000's "N" and "A" The "A" is a standby for the "N".
Both of these servers have smart UPS's that tell the HP to shutdown if the power fails. We have Linux serves hooked to thses UPS's too. How can I get the HP-UX boxes to tell the Linux(RedHat) to shutdown before they shutdown?
The Linux boxes reboot when the power comes back but they always are flaky and have to be rebooted because of not being shutdown orderly.
Maybe there is another way to do this.
Marty

[Chris Vail · Accepted solution]

A lot of this will have to do with the software that you're running on the HP servers that shuts them down. All of the good UPS's have software that allows you to run a script as part of the shutdown procedure. You just need to write that script.

I'm a real fan of secure shell (document attached) that allows one system to seamlessly interact with another. If you'll set up secure shell on both systems, you can cause the one to shutdown the other as part of its own shutdown process. Secure shell does this across the network, encrypted and signed, so there is little chance of a hacker getting control of the system(s).

If your HP box is "foo" and your linux box is "bar", then the command:
ssh bar "shutdown -g0 -y"
(or whatever the appropriate shutdown command is) is all you'll need to shut down the other system.


Chris

[Pete Randall]

Marty,

The only scenario I can think of is to have a script in the normal shutdown process of the HP boxes that does a remsh to the Linux box and tells it to shut down. Unfortunately, I see no method of customizing ups_conf or ups_mond to do anything like this, so we would have to let ups_mond kick off the normal shutdown of the HP box and let the shutdown process communicate with the Linux box. This has the (probably) unwanted affect of having the Linux box go down every time the HP does, though.


Pete

[S.K. Chan]

There are UPSs that have the appropriate software to do this on various platform (PowerChute is one of them). What UPSs are you using ?

[Marty Metras]

Chris,
I have been using ssh with rsync never dawn on me to execute a command on the other box. Thanks for that.
Now maybe you can tell me where the the shutdown script is located on the HP box. I have never had to messed with that. It is HP-UX 11.0.
Marty

[Pete Randall]

Marty,

Shutdown works like this:

Create a script that will execute the desired command and put it in /sbin/init.d (/sbin/init.d/shutdownlinux, for example). Create links to the script for startup and shutdown. In your case you only need the shutdown, but the idea is that these scripts are kicked of when the run level changes during startup or shutdown. The links go in /sbin/rc?.d - where ? is the runlevel you wish to have them executed at. In your case, we would want this to happen right off when we transition from normal multi-user mode (run-level 3, presumably), so the link would go in /sbin/rc2.d (/sbin/rc2.d/K001shutdownlinux,
for example - the K identifies it as a "Kill" script and the sequence number determines the order in which it runs). If you needed to pass any environment paramaters to your startup or shutdown script they would go in a file in /etc/rc.config.d which gets sourced by the /sbin/init.d script (/etc/rc.config.d/shutdownlinux, for example).

Take a look at the existing scripts in /sbin/init.d for guidance. Yours should be much simpler but you'll get the idea.


Pete

[John Dvorchak]

With all due respect to Pete ever time the HPUX box went down the Linux box would also. That may not be what you desire.

As S.K. asked, what UPS are you using. Since it obviously has some shutdown type scripts, perhaps you could swlist -l file to find them. Lets say you have a Power Chute then you can look for:

swlist |grep -i power

you will have to exchange "power" with what ever word you think will match. Then once you do that do a swlist -l file to see what files it put on the HPUX box:

swlist -l file Power.Chute as an example. Good luck.

[Steven E. Protter]

SCM might be an answer for you.

The downside is its got lots of dependencies, and I actually like Chris Vail's answer better than mine.

Link.

I'm linking to a search because I don't know which release is relavent to you.

http://www.software.hp.com/cgi-bin/swdepot_parser.cgi/cgi/displayProducts.pl?group_type=search&group_name=SCM&search_free=1&search_trial=1&search_buy=1

SEP

[James A. Donovan]

You say you have "smart UPS's". Are they actually SmartUPS UPSes from APC?

APC has a pretty nice java-based software called PowerChute, that is available for HP-UX, Linux, NT, etc...

I use it to protect my servers from PG&E's rolling blackouts....

http://www.apcc.com

[Marty Metras]

Sorry I didn't get back to you yesterday. Has issues come up.
Anyway, The one UPS uses the generic UPS control in SAM. The other one use UPS Manager II. It is is a good point about putting the scrip in the shutdown scrip. It would shutdown the Linux box all the time. This means I need something in my script that checks the UPS's power out flag. If true then send the command to shutdown the Linux box.
New question.
Where should I start looking so I can capture that flag?
Marty

[John Dvorchak]

Marty don't assign me points for this. But you may want to start a different thread with the new question.

But as an idea, maybe you could edit the script that shuts down the system and have it create a file somewhere like in /tmp. Have the shutdown script look for that file. On startup you could add an entry that removed that file so it would only be there in the event of a UPS issued shutdown. Another idea would be to look at syslog.log or maybe look at /etc/shutdownlog but that files retains who/what shutdown the system until it is cleaned. Maybe look for todays date and grep that from /etc/shutdownlog.

[Pete Randall]

Marty (and John),

I just wanted to point out that I did mention the shutdown issue in my first post:

"This has the (probably) unwanted affect of having the Linux box go down every time the HP does, though. "



Since you say you have UPS Manager II, I would suggest you look into configuring it. You can set up some pretty elaborate shutdown scenarios if you take full advantage of the software. Consult the "HP UPS Manager II" documentation. I'm searching for a link to it, but haven't found one as yet.


Pete

[Marty Metras]

Ok I just found something.
the shutdownlog does not show the UPS shuting the system down. On the other hand the syslog.log shows and error "...not been connected correctly...".
On the other hand, on the other box, there are message in syslog.log when the box in running on the UPS and when it is on the AC power.
This tell me it is posable to send a message to a file that I monitor and do something when it is on the UPS.
I have to figtue what is going on with the UPS on the other one. It is 1600 miles from here.
I will start an new questions using the information I learned here.
Thanks for your help.
Marty