System Administration
cancel
Showing results for 
Search instead for 
Did you mean: 

Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

SOLVED
Go to solution
PHSC
Frequent Advisor

Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

Greetigns Chaps.  Over the last couple of days we have been having event logs generated for two physical Enclousures and four hard disks.  Thing is, all the hardware in the server is online.  All is CLAIMED.  There are no stale extents on any disks and I can vgsync the volume groups with no issues.   All the cables and connections are secure.  No warning lights on the back of the server.  The timeline of events is thus:

 

ia64 hp server rx6600

HP-UX host B.11.31 U ia64 unlimited-user license

 

Sunday May 5

 

Disk at hardware path 64000/0xfa00/0x15 : Media error

 

Tuesday May 07

 

Enclosure at hardware path 64000/0xfa00/0x1f: Hardware Failure.

Enclosure at hardware path 64000/0xfa00/0x1e: Hardware Failure.

Enclosure at hardware path 64000/0xfa00/0x20: Hardware Failure.

Enclosure at hardware path 64000/0xfa00/0x21: Hardware Failure.

 

Disk at hardware path 64000/0xfa00/0xa : Drive is not responding.

Disk at hardware path 64000/0xfa00/0x8 : Drive is not responding.

 

Wednesday May 08

 

Enclosure at hardware path 64000/0xfa00/0x1e: Hardware Failure.

Enclosure at hardware path 64000/0xfa00/0x20: Hardware Failure.

Disk at hardware path 64000/0xfa00/0x6 : Drive is not responding.

 

Example error logs:

 

Event Time..........: Wed May  8 05:10:59 2013

Severity............: CRITICAL

Monitor.............: msamon_sas

Event #.............: 404

Summary: Enclosure at hardware path 64000/0xfa00/0x1e: Hardware Failure.

Description of Error:  The enclosure services device is not responding.

Probable Cause / Recommended Action: The enclosure services controller has failed or is no longer connected to the host. Check the connection, cables and controller card.

 

Event Time..........: Wed May  8 05:10:59 2013

Severity............: CRITICAL

Monitor.............: disk_em

Event #.............: 3

Summary:  Disk at hardware path 64000/0xfa00/0x6 : Drive is not responding.

Description of Error: The drive failed to respond to a Test Unit Ready command.

Probable Cause / Recommended Action: Check cables, power supply, ensure the drive is powered ON, and if needed replace the drive.

11 REPLIES
PHSC
Frequent Advisor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

Both Power Supply's are OK.  Nothing too nasty looking in the ILO log files.

 

I'm wondering if a suspect dodgy disk could be startign to cause trouble...even though all disks are currently online..?  The four disks have alt links and the hardware giving off errors is in bold below:

 

ioscan

H/W Path     Class                                              Description

===========================================================================

             root

0            ioa                                                System Bus Adapter (4030)

0/0              ba                                             Local PCI-X Busapter122e)

0/0/1/0                tty                                      PCIclass(255,0)(103c1303)

0/0/1/1                tty                                      PCI SimpleComm (03c1302)

0/0/1/2                tty                                      PCI Serial (103c1048)

0/0/2/0                usb                                      NEC OHCI Controller

0/0/2/1                usb                                      NEC OHCI Controller

0/0/2/2                usb                                      NEC EHCI Controller

0/0/2/2.2                 usbms                                 USB Mass Storage [0]

0/1              ba                                             Local PCI-XBusAdapter22e)

0/1/1               slot                                        PCI Slot

0/1/2               slot                                        PCI Slot

0/2              ba                                             Local PCI-XBusAdapter22e)

0/2/1               slot                                        PCI Slot

0/3              ba                                             Local Bus Adapter

0/3/0/0                ba                                       PCItoPCI Bridge

0/3/0/0/0                 slot                                  PCI Slot

0/3/0/0/0/0                  escsi_ctlr                         HP  PCI-E SAS MPT Adapter

0/3/0/0/0/0.0.0                                     ext_bus SAS Device Interface

0/3/0/0/0/0.0.0.1                                      target

0/3/0/0/0/0.0.0.1.0                                       tape HP    Ultrium 4-SCSI

0/4              ba                                             Local PCI-X BusAdapter2e)

0/4/1               slot                                        PCI Slot

0/4/2               slot                                        PCI Slot

0/4/2/0                lan             HP AB352-60003 PCI/PCI-X1000Base-T Dual-portCore

0/4/2/1                lan             HP AB352-6000 PCI/PCI-X 1000Base-T Dual-portCore

0/5              ba                                             Local PCI-X Busdapter 2e)

0/5/1               slot                                        PCI Slot

0/6              ba                                             Local Bus Adapter

0/6/0/0                ba                                       PCItoPCI Bridge

0/6/0/0/0/0                  ba                                 PCItoPCI Bridge

0/6/0/0/0/0/2/0                                     ba PCItoPCI Bridge

0/6/0/0/0/0/2/0/0                                      slot PCI Slot

0/6/0/0/0/0/2/0/0/0                                  escsi_ctlr HP  PCI-E SAS MPT Adapter

0/6/0/0/0/0/2/0/0/0.0.1                                  ext_bus SAS DeviceInterface

0/6/0/0/0/0/2/0/0/0.0.1.4                 target

0/6/0/0/0/0/2/0/0/0.0.1.4.0               disk HP DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.5                 target

0/6/0/0/0/0/2/0/0/0.0.1.5.0                  diskHP   DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.7                 target

0/6/0/0/0/0/2/0/0/0.0.1.7.0                  disk HP  DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.8                 target

0/6/0/0/0/0/2/0/0/0.0.1.8.0                  disk HP  DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.9                 target

0/6/0/0/0/0/2/0/0/0.0.1.9.0                  disk HP  DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.10                target

0/6/0/0/0/0/2/0/0/0.0.1.10.0                 ctl HP      MSA60

0/6/0/0/0/0/2/0/0/0.0.1.11                target

0/6/0/0/0/0/2/0/0/0.0.1.11.0                 disk HP DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.12                target

0/6/0/0/0/0/2/0/0/0.0.1.12.0                 disk HP  DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.13                target

0/6/0/0/0/0/2/0/0/0.0.1.13.0                 disk HP DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.14                target

0/6/0/0/0/0/2/0/0/0.0.1.14.0                 diskHP DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.1.15                target

0/6/0/0/0/0/2/0/0/0.0.1.15.0                 disk HP  DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.2                 ext_bus SAS Device Interface

0/6/0/0/0/0/2/0/0/0.0.2.0                  target

0/6/0/0/0/0/2/0/0/0.0.2.0.0                   disk HP      DF0300B8053

0/6/0/0/0/0/2/0/0/0.0.2.3                  target

0/6/0/0/0/0/2/0/0/0.0.2.3.0                   ctl HP      MSA60

0/6/0/0/0/0/2/0/0/0.0.2.4                  target

0/6/0/0/0/0/2/0/0/0.0.2.4.0                   disk HP      EF0300FARMU

0/6/0/0/0/0/2/0/0/0.0.2.5                  target

0/6/0/0/0/0/2/0/0/0.0.2.5.0                   disk HP      EF0300FARMU

0/6/0/0/0/0/2/0/0/0.0.2.6                  target

0/6/0/0/0/0/2/0/0/0.0.2.6.0                   disk HP      EF0300FARMU

0/6/0/0/0/0/2/0/0/0.0.2.7                  target

0/6/0/0/0/0/2/0/0/0.0.2.7.0                   disk HP      EF0300FARMU

0/6/0/0/0/0/2/0/0/0.0.2.8                  target

0/6/0/0/0/0/2/0/0/0.0.2.8.0                   disk HP      EF0300FARMU

0/6/0/0/0/0/4/0                                     ba PCItoPCI Bridge

0/6/0/0/0/0/4/0/0                                      slot PCI Slot

0/6/0/0/0/0/4/0/0/0                     xt_bus PCIe SAS SmartArray P400 RAID Controller

0/6/0/0/0/0/4/0/0/0.0                                        target

0/6/0/0/0/0/4/0/0/0.0.0                      diskHP      LOGICAL VOLUME

0/7              ba                                             Local Bus Adapter

0/7/0/0                ba                                       PCItoPCI Bridge

0/7/0/0/0                 slot                                  PCI Slot

0/7/0/0/0/0                  escsi_ctlr                 HP  PCI-E SAS MPT Adapter

0/7/0/0/0/0.0.1                                     ext_bus SAS Device Interface

0/7/0/0/0/0.0.1.2                                      target

0/7/0/0/0/0.0.1.2.0                       disk HP      DF0300B8053

0/7/0/0/0/0.0.1.3                      target

0/7/0/0/0/0.0.1.3.0                       disk HP      DF0300B8053

0/7/0/0/0/0.0.1.4                      target

0/7/0/0/0/0.0.1.4.0                       disk HP      DF0300B8053

0/7/0/0/0/0.0.1.6                      target

0/7/0/0/0/0.0.1.6.0                       disk HP      DF0300B8053

0/7/0/0/0/0.0.1.7                      target

0/7/0/0/0/0.0.1.7.0                       disk HP      DF0300B8053

0/7/0/0/0/0.0.1.9                      target

0/7/0/0/0/0.0.1.9.0                       disk HP      DF0300B8053

0/7/0/0/0/0.0.1.10                     target

0/7/0/0/0/0.0.1.10.0                      ctl HP      MSA60

0/7/0/0/0/0.0.1.12                                     target

0/7/0/0/0/0.0.1.12.0                                      disk HP      DF0300B8053

0/7/0/0/0/0.0.1.14                                     target

0/7/0/0/0/0.0.1.14.0                                      disk HP      DF0300B8053

0/7/0/0/0/0.0.1.15                                     target

0/7/0/0/0/0.0.1.15.0                                      disk HP      DF0300B8053

0/7/0/0/0/0.0.2                                     ext_bus SAS Device Interface

0/7/0/0/0/0.0.2.0                                      target

0/7/0/0/0/0.0.2.0.0                                       disk HP      DF0300B8053

0/7/0/0/0/0.0.2.2                                      target

0/7/0/0/0/0.0.2.2.0                                       disk HP      DF0300B8053

0/7/0/0/0/0.0.2.3                                      target

0/7/0/0/0/0.0.2.3.0                                       ctl HP      MSA60

0/7/0/0/0/0.0.2.4                                      target

0/7/0/0/0/0.0.2.4.0                                       disk HP      EF0300FARMU

0/7/0/0/0/0.0.2.5                                      target

0/7/0/0/0/0.0.2.5.0                                       disk HP      EF0300FARMU

0/7/0/0/0/0.0.2.6                                      target

0/7/0/0/0/0.0.2.6.0                                       disk HP      EF0300FARMU

0/7/0/0/0/0.0.2.7                                      target

0/7/0/0/0/0.0.2.7.0                                       disk HP      EF0300FARMU

0/7/0/0/0/0.0.2.8                                      target

0/7/0/0/0/0.0.2.8.0                             disk HP      EF0300FARMU

120          processor                                          Processor

121          processor                                          Processor

122          processor                                          Processor

123          processor                                          Processor

124          processor                                          Processor

125          processor                                          Processor

126          processor                                          Processor

127          processor                                          Processor

250          ba                                                 Core I/O Adapter

250/0            ipmi                                           IPMI Controller

250/1            tty                                            Built-in RS232C

250/2            acpi_node                                      Acpi Hardware

255/0            iscsi                                          iSCSI Virtual Node

255/1            usbmsvbus                                      USB Mass Storage

255/1/0             ext_bus                                     USB Mass Storage SCSI

255/1/0.0              target

255/1/0.0.0               disk                                  TEAC    DVD-ROMDW-224EV

Torsten.
Acclaimed Contributor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

Could you post a

# ioscan -fnN

please?

Are there external disks?

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
Torsten.
Acclaimed Contributor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

Now I saw the ioscan.

However, I would still prefer the

# ioscan -fnN



You have a MSA60 connected to a P400 smartarray controller?

Please find out the ciss_ device of the controller and post

# saconfig /dev/ciss_

# sautil /dev/ciss_


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
PHSC
Frequent Advisor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

Greetings Torsten.  Yes, there are two banks of 300GB external SAS disks.  The full ioscan command is below:

 

ioscan -fnN

Class        I  H/W Path       Driver       S/W State   H/W Type     Description

=================================================================================

root         0                 root           CLAIMED     BUS_NEXUS

ioa          0  0              sba            CLAIMED     BUS_NEXUS    System Bus Adapter (4030)

ba           0  0/0            lba            CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)

tty          0  0/0/1/0        rmp3f01        CLAIMED     INTERFACE    PCI class(255,0) (103c1303)

tty          1  0/0/1/1        rmp3f01        CLAIMED     INTERFACE    PCI SimpleComm (103c1302)

tty          2  0/0/1/2        asio0          CLAIMED     INTERFACE    PCI Serial (103c1048)

                              /dev/diag/mux2  /dev/mux2       /dev/tty2p0

usb          0  0/0/2/0        hcd            CLAIMED     INTERFACE    NEC OHCIController

usb          1  0/0/2/1        hcd            CLAIMED     INTERFACE    NEC OHCIController

usb          2  0/0/2/2        ehci           CLAIMED     INTERFACE    NEC EHCIController

usbms        0  0/0/2/2.2      ms             CLAIMED     DEVICE       USB MassStorage [0]

ba           1  0/1            lba            CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)

slot         0  0/1/1          pci_slot       CLAIMED     SLOT         PCI Slot

slot         1  0/1/2          pci_slot       CLAIMED     SLOT         PCI Slot

ba           2  0/2            lba            CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)

slot         2  0/2/1          pci_slot       CLAIMED     SLOT         PCI Slot

ba           3  0/3            gh2p           CLAIMED     BUS_NEXUS    Local Bus Adapter

ba           4  0/3/0/0        PCItoPCI       CLAIMED     BUS_NEXUS    PCItoPCIBridge

slot         3  0/3/0/0/0      pci_slot       CLAIMED     SLOT         PCI Slot

escsi_ctlr   1  0/3/0/0/0/0    sasd           CLAIMED     INTERFACE    HP  PCI-E SAS MPT Adapter

                              /dev/sasd1

tgtpath     75  0/3/0/0/0/0.0x5001438010be7a58              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5001438010be7a58

lunpath     76  0/3/0/0/0/0.0x5001438010be7a58.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for tape3

ba           5  0/4            lba            CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)

slot         4  0/4/1          pci_slot       CLAIMED     SLOT         PCI Slot

slot         5  0/4/2          pci_slot       CLAIMED     SLOT         PCI Slot

lan          0  0/4/2/0        iether         CLAIMED     INTERFACE    HP AB352-60003 PCI/PCI-X 1000Base-T Dual-port Core

lan          1  0/4/2/1        iether         CLAIMED     INTERFACE    HP AB352-60003 PCI/PCI-X 1000Base-T Dual-port Core

ba           6  0/5            lba            CLAIMED     BUS_NEXUS    Local PCI-X Bus Adapter (122e)

slot         6  0/5/1          pci_slot       CLAIMED     SLOT         PCI Slot

ba           7  0/6            gh2p           CLAIMED     BUS_NEXUS    Local Bus Adapter

ba           8  0/6/0/0        PCItoPCI       CLAIMED     BUS_NEXUS    PCItoPCIBridge

ba           9  0/6/0/0/0/0    PCItoPCI       CLAIMED     BUS_NEXUS    PCItoPCIBridge

ba          10  0/6/0/0/0/0/2/0                             PCItoPCI       CLAIMED     BUS_NEXUS    PCItoPCI Bridge

slot         7  0/6/0/0/0/0/2/0/0                           pci_slot       CLAIMED     SLOT         PCI Slot

escsi_ctlr   2  0/6/0/0/0/0/2/0/0/0                         sasd           CLAIMED     INTERFACE    HP  PCI-E SAS MPT dapter

                              /dev/sasd2

tgtpath     48  0/6/0/0/0/0/2/0/0/0.0x5000c50013352efa      estp           CLAIM

ED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013352efa

lunpath     49  0/6/0/0/0/0/2/0/0/0.0x5000c50013352efa.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk36

tgtpath     66  0/6/0/0/0/0/2/0/0/0.0x5000c50013439db6      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013439db6

lunpath     67  0/6/0/0/0/0/2/0/0/0.0x5000c50013439db6.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk39

tgtpath     51  0/6/0/0/0/0/2/0/0/0.0x5000c5001343a2fa      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001343a2fa

lunpath     52  0/6/0/0/0/0/2/0/0/0.0x5000c5001343a2fa.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk50

tgtpath     52  0/6/0/0/0/0/2/0/0/0.0x5000c5001343be6a      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001343be6a

lunpath     53  0/6/0/0/0/0/2/0/0/0.0x5000c5001343be6a.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk40

tgtpath     67  0/6/0/0/0/0/2/0/0/0.0x5000c5001343e5fe      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001343e5fe

lunpath     68  0/6/0/0/0/0/2/0/0/0.0x5000c5001343e5fe.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk41

tgtpath     68  0/6/0/0/0/0/2/0/0/0.0x5000c50013440b32      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013440b32

lunpath     69  0/6/0/0/0/0/2/0/0/0.0x5000c50013440b32.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk51

tgtpath     53  0/6/0/0/0/0/2/0/0/0.0x5000c50013440b72      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013440b72

lunpath     54  0/6/0/0/0/0/2/0/0/0.0x5000c50013440b72.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk42

tgtpath     54  0/6/0/0/0/0/2/0/0/0.0x5000c50013441432      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013441432

lunpath     55  0/6/0/0/0/0/2/0/0/0.0x5000c50013441432.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk43

tgtpath     69  0/6/0/0/0/0/2/0/0/0.0x5000c500134415b2      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c500134415b2

lunpath     70  0/6/0/0/0/0/2/0/0/0.0x5000c500134415b2.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk44

tgtpath     70  0/6/0/0/0/0/2/0/0/0.0x5000c5001344389e      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001344389e

lunpath     71  0/6/0/0/0/0/2/0/0/0.0x5000c5001344389e.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk45

tgtpath     72  0/6/0/0/0/0/2/0/0/0.0x5000c5001345504a      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001345504a

lunpath     73  0/6/0/0/0/0/2/0/0/0.0x5000c5001345504a.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk48

tgtpath     77  0/6/0/0/0/0/2/0/0/0.0x5000c50023a6ccfe      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50023a6ccfe

lunpath     78  0/6/0/0/0/0/2/0/0/0.0x5000c50023a6ccfe.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk47

tgtpath     79  0/6/0/0/0/0/2/0/0/0.0x5000c5003b787ca6      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5003b787ca6

lunpath     72  0/6/0/0/0/0/2/0/0/0.0x5000c5003b787ca6.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk37

tgtpath     80  0/6/0/0/0/0/2/0/0/0.0x5000c5003bc4533e      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5003bc4533e

lunpath     79  0/6/0/0/0/0/2/0/0/0.0x5000c5003bc4533e.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk38

tgtpath     84  0/6/0/0/0/0/2/0/0/0.0x5000c50043c726e2      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50043c726e2

lunpath     45  0/6/0/0/0/0/2/0/0/0.0x5000c50043c726e2.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk46

tgtpath     83  0/6/0/0/0/0/2/0/0/0.0x5000c5004778db4a      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5004778db4a

lunpath     59  0/6/0/0/0/0/2/0/0/0.0x5000c5004778db4a.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for disk49

tgtpath     56  0/6/0/0/0/0/2/0/0/0.0x500143800533c5e5      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x500143800533c5e5

lunpath     57  0/6/0/0/0/0/2/0/0/0.0x500143800533c5e5.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for ctl12

tgtpath     74  0/6/0/0/0/0/2/0/0/0.0x500143800533de65      estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x500143800533de65

lunpath     75  0/6/0/0/0/0/2/0/0/0.0x500143800533de65.0x0  eslpt          CLAIMED     LUN_PATH     LUN path for ctl14

ba          11  0/6/0/0/0/0/4/0                             PCItoPCI       CLAIMED     BUS_NEXUS    PCItoPCI Bridge

slot         8  0/6/0/0/0/0/4/0/0                           pci_slot       CLAIMED     SLOT         PCI Slot

ext_bus      0  0/6/0/0/0/0/4/0/0/0                         ciss           CLAIMED     INTERFACE    PCIe SAS SmartArray P400 RAID Controller

                              /dev/ciss0

tgtpath      0  0/6/0/0/0/0/4/0/0/0.0x0                     estp           CLAIMED     TGT_PATH     parallel_scsi target served by ciss driver, target port id 0x0

lunpath      0  0/6/0/0/0/0/4/0/0/0.0x0.0x0                 eslpt          CLAIMED     LUN_PATH     LUN path for ctl0

lunpath      1  0/6/0/0/0/0/4/0/0/0.0x0.0x4000000000000000  eslpt          CLAIMED     LUN_PATH     LUN path for disk1

ba          12  0/7            gh2p           CLAIMED     BUS_NEXUS    Local Bus Adapter

ba          13  0/7/0/0        PCItoPCI       CLAIMED     BUS_NEXUS    PCItoPCIBridge

slot         9  0/7/0/0/0      pci_slot       CLAIMED     SLOT         PCI Slot

escsi_ctlr   3  0/7/0/0/0/0    sasd           CLAIMED     INTERFACE    HP  PCI-E SAS MPT Adapter

                              /dev/sasd3

PHSC
Frequent Advisor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

tgtpath     57  0/7/0/0/0/0.0x5000c50013352ef9              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013352ef9

lunpath     58  0/7/0/0/0/0.0x5000c50013352ef9.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk36

tgtpath     39  0/7/0/0/0/0.0x5000c50013439db5              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013439db5

lunpath     42  0/7/0/0/0/0.0x5000c50013439db5.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk39

tgtpath     60  0/7/0/0/0/0.0x5000c5001343a2f9              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001343a2f9

lunpath     60  0/7/0/0/0/0.0x5000c5001343a2f9.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk50

tgtpath     61  0/7/0/0/0/0.0x5000c5001343be69              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001343be69

lunpath     64  0/7/0/0/0/0.0x5000c5001343be69.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk40

tgtpath     40  0/7/0/0/0/0.0x5000c5001343e5fd              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001343e5fd

lunpath     40  0/7/0/0/0/0.0x5000c5001343e5fd.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk41

tgtpath     41  0/7/0/0/0/0.0x5000c50013440b31              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013440b31

lunpath     41  0/7/0/0/0/0.0x5000c50013440b31.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk51

tgtpath     62  0/7/0/0/0/0.0x5000c50013440b71              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013440b71

lunpath     61  0/7/0/0/0/0.0x5000c50013440b71.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk42

tgtpath     63  0/7/0/0/0/0.0x5000c50013441431              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013441431

lunpath     62  0/7/0/0/0/0.0x5000c50013441431.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk43

tgtpath     42  0/7/0/0/0/0.0x5000c500134415b1              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c500134415b1

lunpath     43  0/7/0/0/0/0.0x5000c500134415b1.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk44

tgtpath     43  0/7/0/0/0/0.0x5000c5001344389d              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5001344389d

lunpath     44  0/7/0/0/0/0.0x5000c5001344389d.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk45

tgtpath     45  0/7/0/0/0/0.0x5000c50013455049              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50013455049

lunpath     46  0/7/0/0/0/0.0x5000c50013455049.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk48

tgtpath     76  0/7/0/0/0/0.0x5000c50023a6ccfd              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50023a6ccfd

lunpath     77  0/7/0/0/0/0.0x5000c50023a6ccfd.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk47

tgtpath     78  0/7/0/0/0/0.0x5000c5003b787ca5              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5003b787ca5

lunpath     47  0/7/0/0/0/0.0x5000c5003b787ca5.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk37

tgtpath     81  0/7/0/0/0/0.0x5000c5003bc4533d              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5003bc4533d

lunpath     80  0/7/0/0/0/0.0x5000c5003bc4533d.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk38

tgtpath     85  0/7/0/0/0/0.0x5000c50043c726e1              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c50043c726e1

lunpath     74  0/7/0/0/0/0.0x5000c50043c726e1.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk46

tgtpath     82  0/7/0/0/0/0.0x5000c5004778db49              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x5000c5004778db49

lunpath     51  0/7/0/0/0/0.0x5000c5004778db49.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for disk49

tgtpath     65  0/7/0/0/0/0.0x500143800533c5a5              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x500143800533c5a5

lunpath     66  0/7/0/0/0/0.0x500143800533c5a5.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for ctl4

tgtpath     47  0/7/0/0/0/0.0x500143800533de25              estp           CLAIMED     TGT_PATH     sas target served by sasd driver, target port id 0x500143800533de25

lunpath     48  0/7/0/0/0/0.0x500143800533de25.0x0          eslpt          CLAIMED     LUN_PATH     LUN path for ctl10

processor    0  120            processor      CLAIMED     PROCESSOR    Processor

processor    1  121            processor      CLAIMED     PROCESSOR    Processor

processor    2  122            processor      CLAIMED     PROCESSOR    Processor

processor    3  123            processor      CLAIMED     PROCESSOR    Processor

processor    4  124            processor      CLAIMED     PROCESSOR    Processor

processor    5  125            processor      CLAIMED     PROCESSOR    Processor

processor    6  126            processor      CLAIMED     PROCESSOR    Processor

processor    7  127            processor      CLAIMED     PROCESSOR    Processor

ba          14  250            pdh            CLAIMED     BUS_NEXUS    Core I/OAdapter

ipmi         0  250/0          ipmi           CLAIMED     INTERFACE    IPMI Controller

                              /dev/ipmi

tty          3  250/1          asio0          CLAIMED     INTERFACE    Built-inRS232C

                              /dev/diag/mux3  /dev/mux3       /dev/tty3p0

acpi_node    0  250/2          acpi_node      CLAIMED     INTERFACE    Acpi Hardware

usbmsvbus    0  64000/0x0      mass_storage   CLAIMED     VIRTBUS      USB MassStorage

escsi_ctlr   0  64000/0x0/0x0  usb_ms_scsi    CLAIMED     INTERFACE    USB MassStorage Virt Ctlr

tgtpath      1  64000/0x0/0x0.0x0                           estp           CLAIMED     TGT_PATH     usb target served by usb_ms_scsi driver, target port id 0x0

lunpath      2  64000/0x0/0x0.0x0.0x0                       eslpt          CLAIMED     LUN_PATH     LUN path for disk3

iscsi        0  64000/0x2      iscsi          CLAIMED     VIRTBUS      iSCSI Virtual Root

escsi_ctlr   4  64000/0x2/0x0  isvctlr        CLAIMED     INTERFACE    iSCSI Virtual Controller

esvroot      0  64000/0xfa00   esvroot        CLAIMED     VIRTBUS      Escsi virtual root

ctl          0  64000/0xfa00/0x0                            esctl          CLAIMED     DEVICE       HP      P400

                              /dev/pt/pt0

disk         1  64000/0xfa00/0x1                            esdisk         CLAIMED     DEVICE       HP      LOGICAL VOLUME

                              /dev/disk/disk1      /dev/rdisk/disk1

                              /dev/disk/disk1_p1   /dev/rdisk/disk1_p1

                              /dev/disk/disk1_p2   /dev/rdisk/disk1_p2

                              /dev/disk/disk1_p3   /dev/rdisk/disk1_p3

disk         3  64000/0xfa00/0x2                            esdisk         CLAIMED     DEVICE       TEAC    DVD-ROM DW-224EV

                              /dev/disk/disk3   /dev/rdisk/disk3

disk        36  64000/0xfa00/0x6                            esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk36   /dev/rdisk/disk36

disk        49  64000/0xfa00/0x8                            esdisk         CLAIMED     DEVICE       HP      EF0300FARMU

                              /dev/disk/disk49   /dev/rdisk/disk49

disk        39  64000/0xfa00/0x9                            esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk39   /dev/rdisk/disk39

disk        40  64000/0xfa00/0xa                            esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk40   /dev/rdisk/disk40

disk        41  64000/0xfa00/0xb                            esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk41   /dev/rdisk/disk41

disk        42  64000/0xfa00/0xc                            esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk42   /dev/rdisk/disk42

disk        43  64000/0xfa00/0xd                            esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk43   /dev/rdisk/disk43

disk        44  64000/0xfa00/0xe                            esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk44   /dev/rdisk/disk44

disk        45  64000/0xfa00/0xf                            esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk45   /dev/rdisk/disk45

disk        37  64000/0xfa00/0x11                           esdisk         CLAIMED     DEVICE       HP      EF0300FARMU

                              /dev/disk/disk37   /dev/rdisk/disk37

disk        48  64000/0xfa00/0x12                           esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk48   /dev/rdisk/disk48

disk        46  64000/0xfa00/0x13                           esdisk         CLAIMED     DEVICE       HP      EF0300FARMU

                              /dev/disk/disk46   /dev/rdisk/disk46

disk        50  64000/0xfa00/0x14                           esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk50   /dev/rdisk/disk50

disk        51  64000/0xfa00/0x15                           esdisk         CLAIMED     DEVICE       HP      DF0300B8053

                              /dev/disk/disk51   /dev/rdisk/disk51

ctl         10  64000/0xfa00/0x1e                           esctl          CLAIMED     DEVICE       HP      MSA60

                              /dev/pt/pt10

ctl         12  64000/0xfa00/0x1f                           esctl          CLAIMED     DEVICE       HP      MSA60

                              /dev/pt/pt12

ctl          4  64000/0xfa00/0x20                           esctl          CLAIMED     DEVICE       HP      MSA60

                              /dev/pt/pt4

ctl         14  64000/0xfa00/0x21                           esctl          CLAIMED     DEVICE       HP      MSA60

                              /dev/pt/pt14

tape         3  64000/0xfa00/0x22                           estape         CLAIMED     DEVICE       HP      Ultrium 4-SCSI

                              /dev/rtape/tape3_BEST    /dev/rtape/tape3_BESTn

                              /dev/rtape/tape3_BESTb   /dev/rtape/tape3_BESTnb

disk        47  64000/0xfa00/0x26                           esdisk         CLAIMED     DEVICE       HP      EF0300FARMU

                              /dev/disk/disk47   /dev/rdisk/disk47

disk        38  64000/0xfa00/0x27                           esdisk         CLAIMED     DEVICE       HP      EF0300FARMU

                              /dev/disk/disk38   /dev/rdisk/disk38

PHSC
Frequent Advisor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

ll /dev/ciss*
crw-rw-rw-   1 bin        bin          6 0x000000 Sep 18  2009 /dev/ciss0

saconfig /dev/ciss0 
******************** SmartArray RAID Controller /dev/ciss0 ********************
 
Auto-Fail Missing Disks at Boot     = disabled
Cache Configuration Status          = cache enabled
Cache Ratio                         = 25% Read / 75% Write
 
---------- PHYSICAL DRIVES ----------
 
Location  Ct Enc Bay       WWID           Size       Status
 
Internal  2I   1   8  0x5000c5001380fbbd  73.4 GB    OK
Internal  2I   1   7  0x5000c5001381046d  73.4 GB    OK
 
---------- LOGICAL DRIVE 0 ----------
 
Device File     = c0t0d0
RAID Level      = 1+0
Size            = 69973 MB
Stripe Size     = 128 KB
Status          = OK
 
Participating Physical Drive(s):
 
Ct  Enc  Bay         WWID
2I    1    8  0x5000c5001380fbbd
2I    1    7  0x5000c5001381046d
 
Participating Spare Drive(s):
 
None

PHSC
Frequent Advisor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

sautil /dev/ciss0
 
******************************************************************************
****                                                                      ****
****             S A U T I L   S u p p o r t   U t i l i t y              ****
****                                                                      ****
****             for the HP SmartArray RAID Controller Family             ****
****                                                                      ****
****                           version A.02.14                            ****
****                                                                      ****
****  (C) Copyright 2003-2007 Hewlett-Packard Development Company, L.P.   ****
******************************************************************************
 
 
 
---- DRIVER INFORMATION ------------------------------------------------------
 
  Driver State........................ READY
 
---- CONTROLLER INFORMATION --------------------------------------------------
 
  Controller Product Number........... P400
  Controller Product Name............. HP PCIe SmartArray P400
  Hardware Path....................... 0/6/0/0/0/0/4/0/0/0
  Serial Number....................... PAFGL0T9SXS01V
  Device File......................... /dev/ciss0
  Hardware Revision................... 'E'
  Boot Block Revision................. 0.03
  Firmware Revision (running)......... 5.22
  Firmware Revision (in ROM).......... 5.22
  Firmware Revision (inactive)........ 5.22
  # of Logical Drives................. 1
  # of Physical Disks Configured...... 2
  # of Physical Disks Detected........ 2
  Logical Drive Rebuild Priority...... 0   (low)
  Array Capacity Expansion Priority... 0   (low)
  Auto-Fail Missing Disks at Boot..... disabled
  SCSI Transfer Detection Mode........ Auto Detect
  Controller Path Selection Method.... default
  Max # of Logical Drives............. 2
 
 
---- ARRAY ACCELERATOR (CACHE) INFORMATION -----------------------------------
 
  Array Accelerator Board Present?.... yes
  Cache Configuration Status.......... cache enabled
  Cache Ratio......................... 25% Read / 75% Write
  Total Cache Size (MB)............... 208
    Read Cache........................ 052
    Write Cache....................... 156
    Transfer Buffer................... 000
  Battery Pack Count.................. 1
  Battery Status (pack #1)............ ok
 
 
---- LOGICAL DRIVE SUMMARY ---------------------------------------------------
 
       #   RAID      Size        Status
 
       0   1+0       69973 MB    OK
 
 
---- SAS/SATA DEVICE SUMMARY -------------------------------------------------
 
  Location  Ct Enc Bay       WWID           Type        Capacity Status
 
  internal  2I   1   8  0x5000c5001380fbbd  DISK        73.4 GB  OK
  internal  2I   1   7  0x5000c5001381046d  DISK        73.4 GB  OK
 
 
---- SAS/SATA ENCLOSURE SUMMARY ----------------------------------------------
 
  Location  Ct   Enc  Expander_count    Bay_count     SEP_count
 
  internal  1I   1    0                 4             1
  internal  2I   1    0                 4             1
 
 
---- LOGICAL DRIVE 0 ---------------------------------------------------------
 
  Logical Drive Device File........... ct0d0
  Fault Tolerance Mode................ RAID 1+0 (Disk Mirroring)
  Logical Drive Size.................. 69973 MB
  Logical Drive Status................ OK
  # of Participating Physical Disks... 2
 
  Participating Physical Disk(s)...... Ct:Enc:Bay:WWID
                                       2I:1:8:0x5000c5001380fbbd
                                       2I:1:7:0x5000c5001381046d
 
  Participating Spare Disk(s)......... Ct:Enc:Bay:WWID
                                       none
 
  Stripe Size......................... 128 KB
  Logical Drive Cache Status.......... cache enabled
  Configuration Signature............. 0xAAB3C0D4
  Media Exchange Detected?............ no
 
 
---- SAS/SATA DEVICE 2I:1:8:0x5000c5001380fbbd [DISK] ------------------------
 
  Connector Location............................... internal
  Connector........................................ 2I
  Enclosure........................................ 1
  Bay.............................................. 8
  WWID............................................. 0x5000c5001380fbbd
  Device Type...................................... DISK
  Disk Capacity.................................... 73.4 GB
  Device Status.................................... OK
  Device Vendor ID................................. HP
  Device Product ID................................ DH072BB978
  Device Serial Number............................. 3PD2JY6L00009003LG4E
  Device Firmware Version.......................... HPD9
  Reserved Area (cfg/status info).................. 33554.4 KB (33.6 MB)
  Block Size (bytes/sector)........................ 512 bytes
  M&P Data Stamped?................................ yes
  Last Failure Reason.............................. none
 
  Physical Disk Flags:
    Disk present and operational................... yes
    Non-disk device detected....................... no
 
    S.M.A.R.T. supported........................... yes
    S.M.A.R.T. errors (in factory M&P data)........ no
    S.M.A.R.T. enabled............................. yes
    S.M.A.R.T. errors (in powerup M&P data)........ no
    Attached to external connector................. no
    Configured in a logical drive.................. yes
    Configured as a spare disk..................... no
    Disk write cache enabled at spin up............ no
 
    Supports redundant controller operation........ yes
    Disk write cache enabled in current page....... no
    Disk write cache disabled in default page...... yes
 
 
 
 
---- SAS/SATA DEVICE 2I:1:7:0x5000c5001381046d [DISK] ------------------------
 
  Connector Location............................... internal
  Connector........................................ 2I
  Enclosure........................................ 1
  Bay.............................................. 7
  WWID............................................. 0x5000c5001381046d
  Device Type...................................... DISK
  Disk Capacity.................................... 73.4 GB
  Device Status.................................... OK
  Device Vendor ID................................. HP
  Device Product ID................................ DH072BB978
  Device Serial Number............................. 3PD2JYAZ00009003LHEU
  Device Firmware Version.......................... HPD9
  Reserved Area (cfg/status info).................. 33554.4 KB (33.6 MB)
  Block Size (bytes/sector)........................ 512 bytes
  M&P Data Stamped?................................ yes
  Last Failure Reason.............................. none
 
  Physical Disk Flags:
    Disk present and operational................... yes
    Non-disk device detected....................... no
 
    S.M.A.R.T. supported........................... yes
    S.M.A.R.T. errors (in factory M&P data)........ no
    S.M.A.R.T. enabled............................. yes
    S.M.A.R.T. errors (in powerup M&P data)........ no
    Attached to external connector................. no
    Configured in a logical drive.................. yes
    Configured as a spare disk..................... no
    Disk write cache enabled at spin up............ no
 
    Supports redundant controller operation........ yes
    Disk write cache enabled in current page....... no
    Disk write cache disabled in default page...... yes
 
 
 
 
 
******************************************************************************
****                         End of SAUTIL Output                         ****
******************************************************************************

Torsten.
Acclaimed Contributor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

I'm still not sure about the configuration.

Let's try this:

# /usr/sbin/ioscan -kfnd ciss

and a old fashioned

# ioscan -fn

How is the system cabled?

2 internal disks on the P400?

How about the MSA60?

 

 

 

 

 

 

I guess you have the P400 in slot 3 and another 2 SAS controllers in slots 4 and 5.

 

Do you have a single MSA60 in dual domain configuration (2x HBA+2 cables)?


Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
PHSC
Frequent Advisor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

ioscan -kfnd ciss
Class     I  H/W Path  Driver S/W State   H/W Type     Description
=================================================================
ext_bus   0  0/6/0/0/0/0/4/0/0/0  ciss   CLAIMED     INTERFACE    PCIe SAS Smart
Array P400 RAID Controller
                      /dev/ciss0

 

There are two MSA60 disk racks each holding 8 disks.  There are 2 disks in the actual server (root disks I assume)

 

Hmm, i'm not too sure if the cabling is an issue.  I'll log a call with the HW supplier.  Thanks for your help.

Torsten.
Acclaimed Contributor

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

OK, 2 SAS controllers, 2 MSA, not connected to each other?

Since both reporting controller failures, I guess it could be a driver / firmware issue.

You should check if the MSA and the disks within have the latest firmware (should be 2.18 for the MSA).

Hope this helps!
Regards
Torsten.

__________________________________________________
There are only 10 types of people in the world -
those who understand binary, and those who don't.

__________________________________________________
No support by private messages. Please ask the forum!

If you feel this was helpful please click the KUDOS! thumb below!   
PHSC
Frequent Advisor
Solution

Re: Enclosure & Disk Drive Errors...Even Though All HW Is Online..?

A scsi firmware update looks to have done the trick.

 

The old firmware version on the server was:

 

scsiU320-00            B.11.31.0909   PCI-X SCSI U320;Supptd HW=A7173A/AB290A

 

We have updated the firmware to, "scsiU320-00 B.11.31.1109" and rebooted the server.

 

The daily multiple enclosure and disk warnings have since stopped.

 

Thanks