crash error need help

[cheesytime]

Hello,

I tried to find out with q4 what really caused the following crash:

System Name: HP-UX
Node Name: test
Release: B.11.31
Version: U
Model: no
Machine ID: 3184528279
Processors: 8
Architecture: IA-64
Physical Mem: 8381067 pages

This is a 64 Bit Kernel
The system had been up for 4.59 days (39632683 ticks).
Load averages: 0.00 0.01 0.01.

System went down at: Sun Aug 2 03:56:24 2009

+--------------------------------------------+
| Message Buffer |
+--------------------------------------------+
Loaded ACPI revision 2.0 tables.
MMIO on this platform supports Write Coalescing.
Warning: Setting the expanded_node_host_names parameter to 1 will allow administrators to set node and host names larger than 8 and 64 characters/bytes, respectively. It is strongly recommended that all related manpages and documentation be understood before setting larger names. Larger names can cause some applications which use those names to behave incorrectly or fail.


MCA recovery subsystem disabled, not supported on this platform.
Using /stand/ext_ioconfig

Memory Class Setup
-------------------------------------------------------------------------
Class Physmem Lockmem Swapmem
-------------------------------------------------------------------------
System : 31139 MB 31139 MB 31139 MB
Kernel : 31139 MB 31139 MB 31139 MB
User : 29875 MB 26485 MB 26589 MB
-------------------------------------------------------------------------

ktracer is off until requested.
Installing Socket Protocol families AF_INET and AF_INET6
64000/0xfa00 esvroot
Kernel EVM initialized
64000/0x0 mass_storage
64000/0x0/0x0 usb_ms_scsi
sec_init(): kernel RPC authentication/security initialization.
secgss_init(): kernel RPCSEC_GSS security initialization.
rpc_init(): kernel RPC initialization.
rpcmod_install(): kernel RPC STREAMS module "rpcmod" installation. ...(driver_install)
NOTICE: nfs_client_pv3_install(): nfs3 File system was registered at index 11.
NOTICE: nfs_client_pv4_install(): nfs4 File system was registered at index 12.
NOTICE: cachefsc_install: cachefs File system was registered at index 14.
btlan_load() Loaded Successfully
0 sba
0/0 lba
0/0/1/0 asio0
0/0/1/1 asio0
0/0/2/0 UsbOhci
0/0/2/1 UsbOhci
0/0/2/2 UsbEhci
0/0/3/0 side_multi
0/0/3/0.0 side
0/0/3/0.1 side
0/0/3/0.0.0x0 estp
0/0/4/0 gvid_core
0/1 lba
0/1/1 pci_slot
Initializing the Ultra320 SCSI Controller at 0/1/1/0. Controller firmware version is 01.03.35.69
0/1/1/0 mpt
Initializing the Ultra320 SCSI Controller at 0/1/1/1. Controller firmware version is 01.03.35.69
0/1/1/1 mpt
0/1/2 pci_slot
0/1/2/0 iether
0/1/2/1 iether
0/2 lba
0/2/1 pci_slot
0/3 lba
0/3/1 pci_slot
0/3/2 pci_slot
0/4 lba
0/4/1 pci_slot
fcd: Claimed HP A6826-60001 2Gb Fibre Channel port at hardware path 0/4/1/0 (FC Port 1 on HBA)
0/4/1/0 fcd
fcd: Claimed HP A6826-60001 2Gb Fibre Channel port at hardware path 0/4/1/1 (FC Port 2 on HBA)
0/4/1/1 fcd
0/4/2 pci_slot
0/1/1/0.0x0 estp
0/1/1/0.0x1 estp
0/1/1/0.0x1.0x0 eslpt
0/1/1/0.1 tgt
0/1/1/0.1.0 sdisk
0/1/1/0.0x0.0x0 eslpt
0/1/1/0.0 tgt
0/1/1/0.0.0 sdisk
0/0/3/0.0.0x0.0x0 eslpt
0/0/3/0.0.0 tgt
0/0/3/0.0.0.0 sdisk
td: claimed Tachyon XL2 Fibre Channel Mass Storage card at 0/4/2/0
0/4/2/0 td
0/5 lba
0/5/1 pci_slot
Initializing the Ultra320 SCSI Controller at 0/5/1/0. Controller firmware version is 01.03.35.69
0/5/1/0 mpt
Initializing the Ultra320 SCSI Controller at 0/5/1/1. Controller firmware version is 01.03.35.69
0/5/1/1 mpt
120 processor
121 processor
122 processor
123 processor
124 processor
125 processor
126 processor
127 processor
250 pdh
250/0 ipmi
250/1 acpi_node
255/1 mass_storage
255/1/0 usb_ms_scsi
64000/0xfa00/0x0 esdisk
64000/0xfa00/0x1 esdisk
64000/0xfa00/0x2 esdisk
Boot device's HP-UX HW path is: 0.1.1.0.1.0

System Console is on the Built-In Serial Interface
iether0: INITIALIZING HP AB352-60001 PCI/PCI-X 1000Base-T Dual-port Core at hardware path 0/1/2/0
iether1: INITIALIZING HP AB352-60001 PCI/PCI-X 1000Base-T Dual-port Core at hardware path 0/1/2/1
AF_INET socket/streams output daemon running, pid 48
afinet_prelink: module installed
Starting the STREAMS daemons-phase 1
LVM: Root VG activated
Swap device table: (start & size given in 512-byte blocks)
entry 0 - major is 64, minor is 0x2; start = 0, size = 16777216
Dump device table: (start & size given in 1-Kbyte blocks)
entry 0000000000000000 - major is 3, minor is 0x1; start = 2349920, size = 8388608
Create STCP device files
Starting the STREAMS daemons-phase 2
$Revision: vmunix: B.11.31_LR FLAVOR=perf
Memory Information:
physical page size = 4096 bytes, logical page size = 4096 bytes
Physical: 33524268 Kbytes, lockable: 25184464 Kbytes, available: 28658200 Kbytes

MCA[0]: MCA has occurred
MCA[0]: MCA Recovery - MCA is not recoverable
MCA[0]: er_check = 2, er_process = 0, skip_rendez = 1
MCA[0]: pass1 IO recovery completed
MCA[0]: total wait time = 1389740 usecs
MCA[0]: Error Severity: fatal, cannot recover
MCA[0]: Processor Error Device Info decode begins
MCA[0]: VALIDATION_BITS = 0x000000000100100f
MCA[0]: PSP = 0x20000000fff21320
MCA[0]: MCA was contained, but not isolated (PSP.us ==1)
MCA/CMC[0]: bus check. info:0x0080000002000040
MCA/CMC[0]: req:N/A res:N/A
MCA/CMC[0]: tgt:N/A ip:N/A
MCA[0]: External bus error, eb = 1
MCA[0]: Bus error status, bsi = 2
MCA[0]: MCA Recovery - cannot recover due to external bus error
MCA[0]: PSI_STATIC_STRUCT.VALID_FIELD_BITS=0x000000000000003f
MCA[0]: Processor Error Device Info decode ends
MCA[1]: Error Severity: fatal, cannot recover
MCA[1]: Processor Error Device Info decode begins
MCA[1]: VALIDATION_BITS = 0x000000000100100f
MCA[1]: PSP = 0x20000000fff21320
MCA[1]: MCA was contained, but not isolated (PSP.us ==1)
MCA/CMC[1]: bus check. info:0x0080000002000040
MCA/CMC[1]: req:N/A res:N/A
MCA/CMC[1]: tgt:N/A ip:N/A
MCA[1]: External bus error, eb = 1
MCA[1]: Bus error status, bsi = 2
MCA[1]: MCA Recovery - cannot recover due to external bus error
MCA[1]: PSI_STATIC_STRUCT.VALID_FIELD_BITS=0x000000000000003f
MCA[1]: Processor Error Device Info decode ends
MCA[2]: Error Severity: fatal, cannot recover
MCA[2]: Processor Error Device Info decode begins
MCA[2]: VALIDATION_BITS = 0x000000000100100f
MCA[2]: PSP = 0x20000000fff21320
MCA[2]: MCA was contained, but not isolated (PSP.us ==1)
MCA/CMC[2]: bus check. info:0x0080000002000040
MCA/CMC[2]: req:N/A res:N/A
MCA/CMC[2]: tgt:N/A ip:N/A
MCA[2]: External bus error, eb = 1
MCA[2]: Bus error status, bsi = 2
MCA[2]: MCA Recovery - cannot recover due to external bus error
MCA[2]: PSI_STATIC_STRUCT.VALID_FIELD_BITS=0x000000000000003f
MCA[2]: Processor Error Device Info decode ends

MCA[2]: Platform Specific Non I/O Error
MCA[2]: Platform Specific Data = N/A
MCA[2]: Error Status val = N/A
MCA[2]: Error Status type = N/A

MCA[2]: Platform Specific Non I/O Error
MCA[2]: Platform Specific Data = N/A
MCA[2]: Error Status val = N/A
MCA[2]: Error Status type = N/A
MCA[3]: Error Severity: fatal, cannot recover
MCA[3]: Processor Error Device Info decode begins
MCA[3]: VALIDATION_BITS = 0x000000000100100f
MCA[3]: PSP = 0x20000000fff21320
MCA[3]: MCA was contained, but not isolated (PSP.us ==1)
MCA/CMC[3]: bus check. info:0x0080000002000040
MCA/CMC[3]: req:N/A res:N/A
MCA/CMC[3]: tgt:N/A ip:N/A
MCA[3]: External bus error, eb = 1
MCA[3]: Bus error status, bsi = 2
MCA[3]: MCA Recovery - cannot recover due to external bus error
MCA[3]: PSI_STATIC_STRUCT.VALID_FIELD_BITS=0x000000000000003f
MCA[3]: Processor Error Device Info decode ends
MCA[4]: Error Severity: fatal, cannot recover
MCA[4]: Processor Error Device Info decode begins
MCA[4]: VALIDATION_BITS = 0x000000000100100f
MCA[4]: PSP = 0x20000000fff21320
MCA[4]: MCA was contained, but not isolated (PSP.us ==1)
MCA/CMC[4]: bus check. info:0x0080000002000040
MCA/CMC[4]: req:N/A res:N/A
MCA/CMC[4]: tgt:N/A ip:N/A
MCA[4]: External bus error, eb = 1
MCA[4]: Bus error status, bsi = 2
MCA[4]: MCA Recovery - cannot recover due to external bus error
MCA[4]: PSI_STATIC_STRUCT.VALID_FIELD_BITS=0x000000000000003f
MCA[4]: Processor Error Device Info decode ends
MCA[5]: Error Severity: fatal, cannot recover
MCA[5]: Processor Error Device Info decode begins
MCA[5]: VALIDATION_BITS = 0x000000000100100f
MCA[5]: PSP = 0x20000000fff21320
MCA[5]: MCA was contained, but not isolated (PSP.us ==1)
MCA/CMC[5]: bus check. info:0x0080000002000040
MCA/CMC[5]: req:N/A res:N/A
MCA/CMC[5]: tgt:N/A ip:N/A
MCA[5]: External bus error, eb = 1
MCA[5]: Bus error status, bsi = 2
MCA[5]: MCA Recovery - cannot recover due to external bus error
MCA[5]: PSI_STATIC_STRUCT.VALID_FIELD_BITS=0x000000000000003f
MCA[5]: Processor Error Device Info decode ends
MCA[6]: Error Severity: fatal, cannot recover
MCA[6]: Processor Error Device Info decode begins
MCA[6]: VALIDATION_BITS = 0x000000000100100f
MCA[6]: PSP = 0x20000000fff21320
MCA[6]: MCA was contained, but not isolated (PSP.us ==1)
MCA/CMC[6]: bus check. info:0x0080000002000040
MCA/CMC[6]: req:N/A res:N/A
MCA/CMC[6]: tgt:N/A ip:N/A
MCA[6]: External bus error, eb = 1
MCA[6]: Bus error status, bsi = 2
MCA[6]: MCA Recovery - cannot recover due to external bus error
MCA[6]: PSI_STATIC_STRUCT.VALID_FIELD_BITS=0x000000000000003f
MCA[6]: Processor Error Device Info decode ends
MCA[7]: Error Severity: fatal, cannot recover
MCA[7]: Processor Error Device Info decode begins
MCA[7]: VALIDATION_BITS = 0x000000000100100f
MCA[7]: PSP = 0x20000000fff21320
MCA[7]: MCA was contained, but not isolated (PSP.us ==1)
MCA/CMC[7]: bus check. info:0x0080000002000040
MCA/CMC[7]: req:N/A res:N/A
MCA/CMC[7]: tgt:N/A ip:N/A
MCA[7]: External bus error, eb = 1
MCA[7]: Bus error status, bsi = 2
MCA[7]: MCA Recovery - cannot recover due to external bus error
MCA[7]: PSI_STATIC_STRUCT.VALID_FIELD_BITS=0x000000000000003f
MCA[7]: Processor Error Device Info decode ends
MCA[0]: Processor State Parameter = 0x20000000fff21320
MCA[0]: uc:0 rc:0 bc:1 tc:0 cc:0 dsize:0
MCA[0]: gr:1 b0:1 b1:1 fp:1 pr:1 br:1 ar:1 rr:1
MCA[0]: tr:1 dr:1 pc:1 cr:1 ex:0 cm:0 rs:1 in:0
MCA[0]: dy:0 pm:0 pi:0 mi:1 tl:0 hd:0 us:1 ci:1
MCA[0]: co:0 sy:0 mn:1 me:0 ra:0 rz:0


CPU status:
-----------
CPU 0 - (monarch)
CPU 1 - (slave) crash event table update Successful
CPU 2 - (slave) crash event table update Successful
CPU 3 - (slave) crash event table update Successful
CPU 4 - (slave) crash event table update Successful
CPU 5 - (slave) crash event table update Successful
CPU 6 - (slave) crash event table update Successful
CPU 7 - (slave) crash event table update Successful


MCA[0]: lid = 0x0000000000000000, br0 = 0xe000000000adeac0
MCA[0]: iip = 0xe000000000adea80, xip = 0xe000000000adea80
MCA[0]: ipsr = 0x0000000800000000, xpsr = 0x00001210086ae01a
MCA[0]: ifa = 0x20000000777fc000, xfs = 0x8000000000000001
MCA[0]: Processor State Parameter (PSP) = 0x20000000fff21320
MCA[0]: rendezvous status information GR11 = 0
MCA[0]: procs MCA'ed = 8
MCA[0]: procs rendezvoused = 0
MCA[0]: procs INIT'ed = 0
MCA[0]: procs unconfigured = 0
MCA[0]: procs not rendezvoused = 0

What seems to be the problem folks?

[Sunny123_1 · Accepted solution]

Hi

The best option is to involve HP in this.Log a case with them and send them necessary logs.

Regards
Sunny

[SoorajCleris]

Hi,

Do you have a ts99 file ?

Regards,
Sooraj

[cheesytime]

where would that file be located?

[Steven E. Protter]

Shalom,

If you have a software contract HP will do an analysis of this information.

They have some magic process by which from the crash dump, they determine the patch that is missing and caused the crash.

Looking at the information doesn't tell me anything. It is a good reason as to why you want to have a software support contract.

SEP

[Sunny123_1]

Hi

ts99 is located at /var/tombstone/ts??

Regards
Sunny

[Roopesh Francis_1]

Andrew,

I would recommend to contact HP response centre for analysis the core dump.

Thanks,
Roopesh

[Dennis Handly]

>I tried to find out with q4 what really caused the following crash:
>MCA[0]: MCA has occurred

This looks like a hardware problem, I don't think q4 will help.
What model rx box do you have?