HPE 9000 and HPE e3000 Servers
1748218 Members
4533 Online
108759 Solutions
New Discussion юеВ

Re: N4000 - Unexpected HPMC

 
Dan Bolton
Frequent Advisor

N4000 - Unexpected HPMC

One of our test servers, an N4000-55, crashes and will not reboot after running Ok for several days. Power it down for a day or two and it boots up and runs for several more days. A forum search for "unexpected HPMC" seems to indicate a hardware problem, but I don't know where to go from here.

I have attached the console log of the HPMC message, taken through the GSP. The chassis logs didn't seem very informative to me, but I can post them if it would help.

Thanks,
Dan
...skid in sideways, chocolate in one hand, martini in the other, totally worn out and screaming, "WOO HOO what a ride!"
15 REPLIES 15
Sameer_Nirmal
Honored Contributor

Re: N4000 - Unexpected HPMC

A file called /var/tombstones/ts99 with valid timestamp in it will help.
Prashanth.D.S
Honored Contributor

Re: N4000 - Unexpected HPMC

Hi Dan,

As suggested above file called as ts99 under /var/tombstones will help us analyze the reason for this HPMC..

Can you attach this file here....

Best Regards,
Prashanth
Dan Bolton
Frequent Advisor

Re: N4000 - Unexpected HPMC

Hi,

Here is the ts99 file.
Thank you for any insight you can give.

-Dan
...skid in sideways, chocolate in one hand, martini in the other, totally worn out and screaming, "WOO HOO what a ride!"
Michael Steele_2
Honored Contributor

Re: N4000 - Unexpected HPMC

Hi

Well you definitely got a HW failure but I can't pin it down. I suggest you have HP look at it. I'm sure they'll spot it in a sec.

Good luck.
Support Fatherhood - Stop Family Law
Michael Steele_2
Honored Contributor

Re: N4000 - Unexpected HPMC

Hi Again:

Please send the following in attach.

cstm<<-EOF
runutil logtool
rs
EOF

ioscan -fnk

tail /etc/shutdownlog

Thanks!
Support Fatherhood - Stop Family Law
cnb
Honored Contributor

Re: N4000 - Unexpected HPMC

Hi Dan,



In addition, the PDC is too old and should be upgraded to have critical fixes installed at some point.



hth,
Dan Bolton
Frequent Advisor

Re: N4000 - Unexpected HPMC

Here is the output you requested.
...skid in sideways, chocolate in one hand, martini in the other, totally worn out and screaming, "WOO HOO what a ride!"
Michael Steele_2
Honored Contributor

Re: N4000 - Unexpected HPMC

10:02 Tue Jun 02 2009. Reboot after panic: , isr.ior = 0'10340007.0'b42c0f8
21:13 Mon Jun 08 2009. Reboot after panic: , isr.ior = 0'10340007.0'b42c0f8
22:19 Mon Jun 08 2009. Reboot after panic: , isr.ior = 0'4340003.0'338ef6b0
12:40 Tue Jun 09 2009. Reboot after panic: , isr.ior = 0'10340743.0'91e46148
22:21 Mon Jun 15 2009. Reboot after panic: , isr.ior = 0'10340003.0'6bdae918
23:28 Mon Jun 15 2009. Reboot after panic: , isr.ior = 0'340757.0'98a637b8
08:36 Wed Jun 17 2009. Reboot after panic: , isr.ior = 0'2401ec.0'26c9a178

You've had 7 panic's since June 2nd.
Support Fatherhood - Stop Family Law
Dan Bolton
Frequent Advisor

Re: N4000 - Unexpected HPMC

Yes, 7 panics (on three occasions) in June. The very first panic was one in December then it ran fine until April, when there were two more. The repeated panics started in May with 4 occurrences of repeating panics (16 panics in May).

As I mentioned, the server will be fine for a few days before failing. After crashing, it will try to reboot, but appears to crash again while booting or shortly thereafter. After several cycles (as many as 10 over one weekend) it hangs during a boot and requires a reset, TOC, or power cycle to restart.

This is a test box, and not currently under HW support, so if I cannot figure it out, we will have to call somebody in.

I assume the next step would be to strip it down to a minimal config (i/o cards, RAM, and CPUs) and gradually add components to find the faulty device. Had this been a 'hard' (or at least a predictable) failure I would have begun that process already, but as it may run for several days before failing, it could be winter by the time I isolate the problem... ;)

I am open to any suggestions.
...skid in sideways, chocolate in one hand, martini in the other, totally worn out and screaming, "WOO HOO what a ride!"