rp5470 random halts

[Matti_Kurkela]

Note that the timestamps in the GSP log are in UTC timezone.

So the loss of power would have happened in 16:17:29 UTC = 18:17:29 Finnish local time.

MK

[Timo J]

Summary:

- Matti: ok with the timestamps, last entry on syslog was at 18:00:43.

- /etc/shutdownlog: last entry was intentional reboot few months ago.

- /var/tombstones: nothing relevant

- syslog: nothing relevant

- /etc/opt/resmon/log: nothing relevant

- GSP errors about DC voltage

And to clarify, system didn't halt like the way of 'shutdown -h', instead of that it just died as fast as if I'd pulled the power cords off.

This system is not high priority so it's not under HP support contract. That's why I'm trying to solve the problem here. But now it's starting to look like I had to call HP to do some HW diagnostics.

[Matti_Kurkela]

You kind of said it yourself - "it just died as fast as if I'd pulled the power cords off".

Have you already excluded the possibility of a power black-out? Maybe a circuit breaker was tripped, then reset? Or if you had an electrician working on-site, maybe there was a little "oops"...?

The GSP processor is not particularily fast - it does not need to be. It gets information from diagnostic buses, which may have a very low data rate. The GSP has NVRAM and some capacitors (or a coin-cell battery) that allow it to store a message about loss of power. But if the entire server around the GSP suddenly loses power, the GSP's internal power is not going to be enough to query what's happening on the AC side of the PSUs.

The GSP error message indicates that the DC power inside the server was not at the proper level to keep the machine running. As a rp5470 has multiple PSUs, one would not expect all of them to fail simultaneously unless there is something wrong with the incoming AC power.

When just one of the power cords of a rp54xx series server is disconnected, the resulting GSP log message looks like this:

ALERT LEVEL: 6 = Boot possible, pending failure - action required

SOURCE: 4 = power
SOURCE DETAIL: 4 = high voltage DC power SOURCE ID: 0
PROBLEM DETAIL: A = failed or disconnected

CALLER ACTIVITY: 4 = monitor STATUS: F
CALLER SUBACTIVITY: 04 = low voltage power supply
REPORTING ENTITY TYPE: 2 = power monitor REPORTING ENTITY ID: 00

MK

[Michael Steele_2]

A) Regarding no entry in /etc/shutdownlog: 90% of the time this indicates a HW problem and not an O/S problem.

B) To check your power module status use 'PS' from GSP as well as the system status also from the GSP 'SS'.

PS : Power Status- display the status of the Power Management Module
This command displays on the console the status of the power management module.
The firmware revision listed is the power management module firmware.

SS : System Status of proc.

[Timo J]

There's another rp5470 connected to same power source as this problem host and it has been running ok for at least two months and never had experienced same kind of problems as other host. So I think that power black-out is out of question in this case.

Also SS & PS reports on GSP are ok.

[Michael Steele_2]

Well this is odd. Your missing something. Next time it happens though take a crash dump, run through q4 and send it up to hp for analysis.

savecrash -rf /dir

Since you're not getting any automatic crashdumps use the above command to dump what's not been overwritten from the command line. Verify that you're set up for dumping properly.

lvlnboot -d

crashconf

Don't know what else to suggest. Sorry.

[Steve Post]

I'll say it again.... U. P. S. Actually the batteries in the UPS.