RP5450 rebooted with few traces left

[mgais]

Hello,

I have a RP5450 server (with HP-UX 11i v1) that the other night has rebooted unexpectedly, leaving very few traces:
- nothing in syslog
- nothing in shutdownlog
- in /var/crash there is no crash file
- in /var/tombstones there is a ts99 with correct Unix timestamp, but reading it, the registers are all zeros and for the HPMC
info it says "no valid timestamps"
- 'last' command reports a "reboot system boot "

A similar event happened about 2 months ago.
Reboot during the night with very few traces left.

Where do I begin to investigate the possible cause? The machine basically is used for SCM tasks.

[Warren_9 · Accepted solution]

hi,

how about the MP console log? any message show on console? and maybe other MP log.

GOOD LUCK!!

[Enrico P.]

Hi,
for the log until the reboot check the /var/adm/syslog/OLDsyslog.log.

For the crash file you need to set the SAVECRASH=1 e SAVECRASH_DIR= in the /etc/rc.config.d/savecrash file.


Enrico

[RAC_1]

Any problems with power supply/UPS?

Did you check console logs? and chasis logs?
Have you configured envd? If yes, anything in there?

[mgais]

@Warren: sorry I'm not sure I understand what you mean with MP console. Nothing is connected to physical console (/dev/console).
I now checked the logs from GSP and there is some event logged there. I've to analyze those.

@Enrico: currently there is no SAVECRASH settings in crashconf, but there is a CRASHCONF_ENABLED=1. Is it the same? Anyway, the server has in some other circumstances dumped a crashfile.

Thanks to both!

[Warren_9]

hi,

MP = GSP, hope some hint from there...

GOOD LUCK!!

[Warren_9]

BTW,

the SAVECRASH is in /etc/rc.config.d/savecrash
not in the crashconf.

[rariasn]

Hi Massimo,

Verify also:

/etc/shutdownlog
/etc/rc.log
/etc/rc.log.old

rgs,

ran

[Enrico P.]

Hi,
the crashconf is for the dump, the savecrash is for the crash file creation.

For to see the mp log:

Log to console:

ctrl-b

user=Admin
password=Admin (default)

sl (show log)

select error log

Enrico

[mgais]

Ok, I have 11 events logged in MP.
The oldest entry is:

ALERT LEVEL: 12 = Software failure

SOURCE: 1 = processor
SOURCE DETAIL: 1 = processor general SOURCE ID: 0
PROBLEM DETAIL: 0 = no problem detail

CALLER ACTIVITY: B = system panic STATUS: 0
CALLER SUBACTIVITY: 80 = implementation dependent
REPORTING ENTITY TYPE: E = HP-UX REPORTING ENTITY ID: 00

0xF8E000C01100B800 00000000 0000B800 type 31 = legacy PA HEX chassis-code
0x58E008C01100B800 00006A04 06130723 type 11 = Timestamp 05/06/2006 19:07:35

Next is an event of the type Software failure/system panic/major change in the system state, at same time.

Then another pair of similar "Software failure" 3 mins later, then six "Non-urgent operator attention required", 1s later, and last two warnings "System blocked waiting for operator input" a couple of minutes later.

Nothing in /etc/rc.log*

About the SAVECRASH, all the parameters there are commented out, and by default I think it means enabled. However, I will explicitely enable, to be on the safe side.