RP2470 shutdown

[Dave Forber]

We currently have a series of RP2470's running 11i for the front end of our ServiceGuard cluster (back end are RP8400 db's).

Over the weekend we had a major incident when the apps servers were powered down. There were no power lights at all on the front display. I had to press the reset switch on the back to bring them back on.

Two questions:

Firstly, if the shutdown command was issued from the command line, would this power the box down to the level of not even showing that power was still connected? Most other servers I know would still display a power connection if nothing else.

Secondly, trying to investigate who may have done this (should it have been from a session), I'm finding that each day logs for last, mail, and su are cleaned out, only giving the current day. There is nothing obvious in the root crontab. Any suggestions where to find what is doing this?

Any advice appreciated.

Thanks.

[Patrick Wallek]

Have a look at the /etc/shutdownlog. If the machine was shutdown via the shutdown command, then it will be logged there.

As to your first question, if you do a 'shutdown -h', that merely halts the system and you get a message similar to "OK to power down or reset system". It does not, as far as I know, actually power the system off.

Now I THINK there may be a GSP command that will allow you to do some power management on some of these systems. I don't recall the GSP commands, but I believe you can power a system off this way.

The other would be if you have these system attached to a UPS and the UPS lost power and the batteries then died cutting power to your systems.

This definitely requires some investigation.

[Pete Randall]

Dave,

Firstly: No, not in any of my experiences

Secondly: How about a power failure? Check /var/adm/syslog/OLDsyslog.log for evidence of a power failure.

If you lost power and the UPS was configured to stay off when the power was restored, this could account for the symptoms you've seen.


Pete

[Patrick Wallek]

The GSP command I was trying to think of is 'PC'. This allows remote Power On/Off of the machine as long as you have some sort of console access (web console, lan console, etc.).

Were things like circuit breakers checked to make sure that you didn't have a breaker trip? Hopefully these are not all on the same breaker.

[Dave Forber]

Cheers guys. There is nothing to suggest this was done legitimately so far. I guess the only outstanding question is whether there are any natural processes that would 'trim' log files each night?

[Patrick Wallek]

No, there are no DEFAULT HP-UX processes that do logfile trimming.

Some add-on products, like ITO/OVO or similar products, might do stuff like that.

One other thing you can try, though it won't necessarily give you specific dates or times, is to look at the GSP's console log. This may give you a bit of an idea as to what happened.

Access the console (web, lan, terminal, whatever) and go into GSP --- -B and log in if necessary. At the GSP prompt type 'CL'. This will display the GSP's console log and show approximately 20KB worth (according to the GSP help on my A500) of text.

Good luck.

[Stefan Stechemesser · Accepted solution]

Hi,

it is not possible to power down the system with a shutdown or reboot or any else command from an OS session.

It is possible to power down the system by logging into the GSP of the server ("pc"). It can also be powered on again with the same command.

I cannot tell you if turned off the server manually, make sure that the GSP is password protected and that nobody has physical access to the server and the console.

The system will power off and remain off after short power failures (< 5 sec). Then you would also have to turn it on manually with the power switch. After longer power failures, the system boots automatically. This was done to prevent from ping pong effects if a shorted circuit is responsible for the power failure.
Maybe you simply had a power problem in your power distribution ?

Another interesting command is the "cl" on the GSP. It displays the x kB of the console log. If the error happens again, you maybe find useful information here.

best regards

Stefan