System rebooting after panic... where to start looking for answers?

[Timothy Czarnik]

Hey all!

I have a K460 that has begun rebooting itself after panics. Once on the 2nd, the 7th, and again last night (not to mention the one last month). /etc/shutdownlog has this:

11:52 Mon Dec 11 2000. Reboot after panic: , isr.ior = 0'10240029.0'2dcb60f8
15:29 Tue Jan 02 2001. Reboot after panic: , isr.ior = 0'0.0'0
01:05 Sun Jan 07 2001. Reboot after panic: , isr.ior = 0'34002b.40000000'f63f2208
06:42 Wed Jan 10 2001. Reboot after panic: , isr.ior = 0'240002.0'6a5a8d94

The syslog mentions NOTHING about these... Where does a prudent sysadmin begin looking for answers? I have started xstm and sort of looked around, but not having really used it before I find myself confused more than anything. I can provide further data if there are some sleuths out there with time to burn...

I appreciate any help!

-Tim

[Rita C Workman · Accepted solution]

Here are a couple things to get you started..
You've done the first..record what the panic said next you need to try to determine hardware/lan/file system or what caused it
In your case the isr.osr is idicative of hardware failure.
1. Check /var/tombstone/ts99
Your looking for problems with hardware, especially a problem with a memory board.

I'd recommend calling in hardware on this one.

But for future panics, it would again depend on the message...for a filesystem panic you would probably want to umount the filesystem and run a full fsck on it.
If you got a coredump at /var/adm/crash...you might want to do the Q4 utility and send it off to HP to resolve why the system did a panic.

Just a couple thoughts,

[melvyn burnard]

The panic messages you have in your shutdownlog show that you have either a hardware failure in the system causing an HPMC, or if the node is part of an MC/ServiceGuard cluster, then it may be a TOC induced by ServiceGuard.
Your first point of call should be to look at the /var/tombstones files, and look at the Timestamp entries in the lates 2 or 3 files, e.g.ts98, ts99. If these show timestamps similar to you panic, you then look for the keywords HPMC in these logs, and pass that on to your hardware maintainer (hopefully HP ;-}).
If this yields no result, or they do not exist, then you either try to use the q4 tool yourself, or get the dump analysed by HP.