Noticed this in syslog.log Mar 22 02:07:08 ed EMS [2933]: ------ EMS Event Notification ------ Value: "CR ITICAL (5)" for Resource: "/storage/events/disks/default/0_12_0_0.8.0" (Thre shold: >= " 3") Execute the following command to obtain event details: /op t/resmon/bin/resdata -R 192217130 -r /storage/events/disks/default/0_12_0_0.8.0 -n 192217133 -a So I ran: /opt/resmon/bin/resdata -R 192217130 -r /storage/events/disks/default/0_12_0_0.8.0 -n 192217133 -a And got this: ARCHIVED MONITOR DATA: Event Time..........: Thu Mar 22 02:07:08 2012 Severity............: CRITICAL Monitor.............: disk_em Event #.............: 100158 System..............: ed.celltech.com Summary: Disk at hardware path 0/12/0/0.8.0 : Hardware failure Description of Error: While attempting to process an I/O request, the device reported a mechanical positioning error. Probable Cause / Recommended Action: The device has experienced a hardware failure. Contact your HP support representative to have the device checked. Additional Event Data: System IP Address...: 192.168.252.108 Event Id............: 0x4f6af9cc00000000 Monitor Version.....: B.01.01 Event Class.........: I/O Client Configuration File...........: /var/stm/config/tools/monitor/default_disk_em.clcfg Client Configuration File Version...: A.01.00 Qualification criteria met. Number of events..: 1 Associated OS error log entry id(s): 0x4f6af9cc00000000 Additional System Data: System Model Number.............: 9000/800/L3000-7x OS Version......................: B.11.11 STM Version.....................: A.38.00 EMS Version.....................: A.03.20 Latest information on this event: http://docs.hp.com/hpux/content/hardware/ems/scsi.htm#100158 v-v-v-v-v-v-v-v-v-v-v-v-v D E T A I L S v-v-v-v-v-v-v-v-v-v-v-v-v Component Data: Physical Device Path...: 0/12/0/0.8.0 Device Class...........: Disk Inquiry Vendor ID......: HP 18.2G Inquiry Product ID.....: ST318453LC Firmware Version.......: HPC3 Serial Number..........: 3JY0973Z00007336L64Y Product/Device Identification Information: Logger ID.........: sdisk Product Identifier: SCSI Disk Product Qualifier.: HP18.2GST318453LC SCSI Target ID....: 0x08 SCSI LUN..........: 0x00 I/O Log Event Data: Driver Status Code..................: 0x0000007E Length of Logged Hardware Status....: 22 bytes. Offset to Logged Manager Information: 24 bytes. Length of Logged Manager Information: 34 bytes. Hardware Status: Raw H/W Status: 0x0000: 00 00 00 02 F0 00 04 00 00 00 10 0A 00 00 00 00 0x0010: 15 01 01 00 00 00 SCSI Status...: CHECK CONDITION (0x02) Indicates that a contingent allegiance condition has occurred. Any error, exception, or abnormal condition that causes sense data to be set will produce the CHECK CONDITION status. SCSI Sense Data: Undecoded Sense Data: 0x0000: F0 00 04 00 00 00 10 0A 00 00 00 00 15 01 01 00 0x0010: 00 00 SCSI Sense Data Fields: Error Code : 0x70 Segment Number : 0x00 Bit Fields: Filemark : 0 End-of-Medium : 0 Incorrect Length Indicator : 0 Sense Key : 0x04 Information Field Valid : TRUE Information Field : 0x00000010 Additional Sense Length : 10 Command Specific : 0x00000000 Additional Sense Code : 0x15 Additional Sense Qualifier : 0x01 Field Replaceable Unit : 0x01 Sense Key Specific Data Valid : FALSE Sense Key Specific Data : 0x00 0x00 0x00 Sense Key 0x04, HARDWARE ERROR, indicates that the device detected a nonrecoverable hardware failure (for example, controller failure, device failure, parity error, etc.) while performing the command or during a self test. The combination of Additional Sense Code and Sense Qualifier (0x1501) indicates: Mechanical positioning error. SCSI Command Data Block: Command Data Block Contents: 0x0000: 28 00 00 00 00 10 00 00 04 00 Command Data Block Fields (10-byte fmt): Command Operation Code...(0x28)..: READ Logical Unit Number..............: 0 DPO Bit..........................: 0 FUA Bit..........................: 0 Relative Address Bit.............: 0 Logical Block Address............: 16 (0x00000010) Transfer Length..................: 4 (0x0004) Manager-Specific Data Fields: Request ID.............: 0x0717657B Data Residue...........: 0x00000800 CDB status.............: 0x00000002 Sense Status...........: 0x00000000 Bus ID.................: 0x07 Target ID..............: 0x08 LUN ID.................: 0x00 Sense Data Length......: 0x12 Q Tag..................: 0x73 Retry Count............: 0 Latest@ # ioscan -fnC disk Class I H/W Path Driver S/W State H/W Type Description ======================================================================= disk 0 0/0/1/1.0.0 sdisk CLAIMED DEVICE HP 36.4GMAM3367MC /dev/dsk/c1t0d0 /dev/rdsk/c1t0d0 disk 1 0/0/1/1.2.0 sdisk CLAIMED DEVICE HP 36.4GMAM3367MC /dev/dsk/c1t2d0 /dev/rdsk/c1t2d0 disk 2 0/0/2/0.0.0 sdisk CLAIMED DEVICE HP 36.4GMAM3367MC /dev/dsk/c2t0d0 /dev/rdsk/c2t0d0 disk 3 0/0/2/0.2.0 sdisk CLAIMED DEVICE HP 36.4GMAM3367MC /dev/dsk/c2t2d0 /dev/rdsk/c2t2d0 disk 4 0/0/2/1.2.0 sdisk CLAIMED DEVICE HP DVD-ROM 305 /dev/dsk/c3t2d0 /dev/rdsk/c3t2d0 disk 5 0/10/0/0.8.0 sdisk CLAIMED DEVICE HP 18.2GST318453LC /dev/dsk/c6t8d0 /dev/rdsk/c6t8d0 disk 6 0/10/0/0.10.0 sdisk CLAIMED DEVICE HP 18.2GST318453LC /dev/dsk/c6t10d0 /dev/rdsk/c6t10d0 disk 7 0/10/0/0.12.0 sdisk CLAIMED DEVICE HP 18.2GST318453LC /dev/dsk/c6t12d0 /dev/rdsk/c6t12d0 disk 8 0/10/0/0.14.0 sdisk CLAIMED DEVICE HP 18.2GST318453LC /dev/dsk/c6t14d0 /dev/rdsk/c6t14d0 Bad disk??? ==> disk 9 0/12/0/0.8.0 sdisk CLAIMED DEVICE HP 18.2GST318453LC /dev/dsk/c7t8d0 /dev/rdsk/c7t8d0 disk 10 0/12/0/0.10.0 sdisk CLAIMED DEVICE HP 18.2GST318453LC /dev/dsk/c7t10d0 /dev/rdsk/c7t10d0 disk 11 0/12/0/0.12.0 sdisk CLAIMED DEVICE HP 18.2GMAS3184NC /dev/dsk/c7t12d0 /dev/rdsk/c7t12d0 disk 12 0/12/0/0.14.0 sdisk CLAIMED DEVICE HP 18.2GST318453LC /dev/dsk/c7t14d0 /dev/rdsk/c7t14d0 # dmesg Apr 1 07:31 ... 8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! lv_readvgdats: Could not read VGDA 1 header & trailer from disk H/W path 0/12/0/0.8.0 (error = 5) LVM: Failed to restore PV 1 to VG 1! LVM: Path (device 0x1f078000) to PV 1 in VG 1 Failed! LVM: vg[1]: pvnum=1 (dev_t=0x1f078000) is POWERFAILED