topic Re: rp4440 / 11.23 / Reboot after panic: Break instruction trap in Operating System - HP-UX

rp4440 / 11.23 / Reboot after panic: Break instruction trap

moonchild — Tue, 11 Mar 2008 19:38:57 GMT

2 nodes cluster in a SG env.

shutdownlog has :
Reboot after panic: Break instruction trap

ts99 has no valid timestamps and it's all zeros.

old syslog has no erros and it is attached

the mp logs has the following:
283 BMC 2 0x2047D69F3C021550 FFFF027000120300 Type-02 127002 1208322
11 Mar 2008 15:03:24
282 SFW 0 2 0x4380100800E01530 000000000096EA48 MC_BR_TO_OS_TOC
11 Mar 2008 14:55:45
281 SFW 3 2 0x4380100803E01510 000000000096EA48 MC_BR_TO_OS_TOC
11 Mar 2008 14:55:45
280 SFW 1 2 0x4380100801E014F0 000000000096EA48 MC_BR_TO_OS_TOC
11 Mar 2008 14:55:45
279 SFW 0 2 0x57800F7300E014D0 2000000000000000 ERR_CPU_CHECK_SUMMARY
11 Mar 2008 14:55:27
278 SFW 3 2 0x57800F7303E014B0 2000000000000000 ERR_CPU_CHECK_SUMMARY
11 Mar 2008 14:55:26
277 SFW 1 2 0x57800F7301E01490 2000000000000000 ERR_CPU_CHECK_SUMMARY
11 Mar 2008 14:55:26
276 OS 2 *3 0x76800C6802E01470 00000000000005E9 PAT_DATA_FIELD_WARNING
11 Mar 2008 14:55:26
275 OS 2 *3 0x78800C6202E01450 A0E028C01100B000 PAT_ENCODED_FIELD_WARNIN
11 Mar 2008 14:55:26

trace.out from q4 gives:
stack trace for event 0
crash event was a panic
can not find unwind or stub descriptor for =c==0x0`43b7f1c0
panic+0x8c
report_trap_or_int_and_panic+0x94
interrupt+0x230
$ihndlr_rtn+0x0

Thanks in advance

Re: rp4440 / 11.23 / Reboot after panic: Break instruction trap

Steven E. Protter — Tue, 11 Mar 2008 20:10:49 GMT

Shalom,

Looks like an application problem.

Bad code. Send the q4 output to HP for analsys, if you need a patch, they will tell you.

Try booting single user and disabling the offending application so at least you have a system.

Check console GSP for HPMC hardware errors.

SEP

Re: rp4440 / 11.23 / Reboot after panic: Break instruction trap

moonchild — Tue, 11 Mar 2008 23:58:28 GMT

S.E.P.

ts99 had no HPMCs and the MP log is showing:

MC_BR_TO_OS_TOC (3 times)

ERR_CPU_CHECK_SUMMARY (3 times):

14:55:45 SFW 0 2
0x57800F7300E014D0 2000000000000000 ERR_CPU_CHECK_SUMMARY

14:55:27 SFW 3 2
0x57800F7303E014B0 2000000000000000 ERR_CPU_CHECK_SUMMARY

14:55:26 SFW 1 2
0x57800F7301E01490 2000000000000000 ERR_CPU_CHECK_SUMMARY

14:55:26 OS 2 *3
0x76800C6802E01470 00000000000005E9 PAT_DATA_FIELD_WARNING

Re: rp4440 / 11.23 / Reboot after panic: Break instruction trap

Sameer_Nirmal — Wed, 12 Mar 2008 03:37:36 GMT

The panic may have been caused by a system/layered s/w or by an application s/w.

The stack trace is in-complete so one can't tell what caused the panic.

I would use crashinfo to get more information about panic. I would also verify the patch level of the system maybe there is patch for a particular s/w running on the system.

The MP event logs are obvious and are the results of the panic/TOC.

Re: rp4440 / 11.23 / Reboot after panic: Break instruction trap

Michael Steele_2 — Wed, 12 Mar 2008 03:56:02 GMT

There are other threads documenting : ERR_CPU_CHECK_SUMMARY. Which is an intermittent CPU error.

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1018377

Gee, one of them is yours moonchild ???? What's up? Same boxes or what?

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1189313

Oh, that was a patch issue. Never mind.

However, Reboot after panic: Break instruction trap, seems to also indicate a flaky CPU.

What's in your EMS logs?

/var/opt/resmon/log/event.log