SCSI errors - NO_HW shows in IOSCAN listing......

[Andy Miller_4]

Hi all,

I have just noticed that we are experiencing some SCSI errors in /var/adm/sys/syslog.log........

For some reason, 1 disk in our SC10 array seems to be showing as NO_HW but also, NO_HW also shows for HP product A5272A............

Here are the results of the IOSCAN..........


# /usr/sbin/ioscan -fn|more
Class I H/W Path Driver S/W State H/W Type Description
============================================================================
root 0 root CLAIMED BUS_NEXUS
ioa 0 0 sba CLAIMED BUS_NEXUS System Bus Adapter (582)
ba 0 0/0 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
lan 0 0/0/0/0 btlan3 CLAIMED INTERFACE HP PCI 10/100Base-TX Core
/dev/diag/lan0 /dev/ether0
ext_bus 0 0/0/1/0 c720 CLAIMED INTERFACE SCSI C896 Ultra Wide LVD
target 0 0/0/1/0.7 tgt CLAIMED DEVICE
ctl 0 0/0/1/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c0t7d0
target 1 0/0/1/0.8 tgt CLAIMED DEVICE
disk 0 0/0/1/0.8.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c0t8d0 /dev/rdsk/c0t8d0
target 2 0/0/1/0.9 tgt CLAIMED DEVICE
disk 1 0/0/1/0.9.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c0t9d0 /dev/rdsk/c0t9d0
target 3 0/0/1/0.10 tgt CLAIMED DEVICE
disk 2 0/0/1/0.10.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c0t10d0 /dev/rdsk/c0t10d0
target 4 0/0/1/0.11 tgt NO_HW DEVICE
disk 3 0/0/1/0.11.0 sdisk NO_HW DEVICE SEAGATE ST318203LC
/dev/dsk/c0t11d0 /dev/rdsk/c0t11d0
target 5 0/0/1/0.14 tgt NO_HW DEVICE
ctl 1 0/0/1/0.14.0 sctl NO_HW DEVICE HP A5272A
/dev/rscsi/c0t14d0
ext_bus 1 0/0/1/1 c720 CLAIMED INTERFACE SCSI C896 Ultra Wide Single-Ended
target 6 0/0/1/1.2 tgt CLAIMED DEVICE
disk 4 0/0/1/1.2.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c1t2d0 /dev/rdsk/c1t2d0
target 7 0/0/1/1.7 tgt CLAIMED DEVICE
ctl 2 0/0/1/1.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c1t7d0
ext_bus 2 0/0/2/0 c720 CLAIMED INTERFACE SCSI C875 Ultra Wide Single-Ended
target 8 0/0/2/0.7 tgt CLAIMED DEVICE
ctl 3 0/0/2/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c2t7d0
ext_bus 3 0/0/2/1 c720 CLAIMED INTERFACE SCSI C875 Ultra Wide Single-Ended
target 9 0/0/2/1.2 tgt CLAIMED DEVICE
Class I H/W Path Driver S/W State H/W Type Description
============================================================================
root 0 root CLAIMED BUS_NEXUS
ioa 0 0 sba CLAIMED BUS_NEXUS System Bus Adapter (582)
ba 0 0/0 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
lan 0 0/0/0/0 btlan3 CLAIMED INTERFACE HP PCI 10/100Base-TX Core
/dev/diag/lan0 /dev/ether0
ext_bus 0 0/0/1/0 c720 CLAIMED INTERFACE SCSI C896 Ultra Wide LVD
target 0 0/0/1/0.7 tgt CLAIMED DEVICE
ctl 0 0/0/1/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c0t7d0
target 1 0/0/1/0.8 tgt CLAIMED DEVICE
disk 0 0/0/1/0.8.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c0t8d0 /dev/rdsk/c0t8d0
target 2 0/0/1/0.9 tgt CLAIMED DEVICE
disk 1 0/0/1/0.9.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c0t9d0 /dev/rdsk/c0t9d0
target 3 0/0/1/0.10 tgt CLAIMED DEVICE
disk 2 0/0/1/0.10.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c0t10d0 /dev/rdsk/c0t10d0
target 4 0/0/1/0.11 tgt NO_HW DEVICE
disk 3 0/0/1/0.11.0 sdisk NO_HW DEVICE SEAGATE ST318203LC
/dev/dsk/c0t11d0 /dev/rdsk/c0t11d0
target 5 0/0/1/0.14 tgt NO_HW DEVICE
ctl 1 0/0/1/0.14.0 sctl NO_HW DEVICE HP A5272A
/dev/rscsi/c0t14d0
ext_bus 1 0/0/1/1 c720 CLAIMED INTERFACE SCSI C896 Ultra Wide Single-Ended
target 6 0/0/1/1.2 tgt CLAIMED DEVICE
disk 4 0/0/1/1.2.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c1t2d0 /dev/rdsk/c1t2d0
target 7 0/0/1/1.7 tgt CLAIMED DEVICE
ctl 2 0/0/1/1.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c1t7d0
ext_bus 2 0/0/2/0 c720 CLAIMED INTERFACE SCSI C875 Ultra Wide Single-Ended
target 8 0/0/2/0.7 tgt CLAIMED DEVICE
ctl 3 0/0/2/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c2t7d0
ext_bus 3 0/0/2/1 c720 CLAIMED INTERFACE SCSI C875 Ultra Wide Single-Ended
target 9 0/0/2/1.2 tgt CLAIMED DEVICE
disk 5 0/0/2/1.2.0 sdisk CLAIMED DEVICE HP DVD-ROM 304
/dev/dsk/c3t2d0 /dev/rdsk/c3t2d0
target 10 0/0/2/1.7 tgt CLAIMED DEVICE
ctl 4 0/0/2/1.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c3t7d0
tty 0 0/0/4/0 asio0 CLAIMED INTERFACE PCI Serial (103c1048)
/dev/GSPdiag1 /dev/mux0 /dev/tty0p1
/dev/diag/mux0 /dev/tty0p0 /dev/tty0p2
tty 1 0/0/5/0 asio0 CLAIMED INTERFACE PCI Serial (103c1048)
/dev/GSPdiag2 /dev/diag/mux1 /dev/mux1 /dev/tty1p1
ba 1 0/1 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
ba 2 0/2 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
ba 3 0/3 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
ba 4 0/4 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
ext_bus 4 0/4/0/0 c720 CLAIMED INTERFACE SCSI C895 Ultra2 Wide Single-Ended
target 11 0/4/0/0.6 tgt CLAIMED DEVICE
tape 0 0/4/0/0.6.0 stape CLAIMED DEVICE HP C1537A
/dev/rmt/0m /dev/rmt/c4t6d0BEST /dev/rmt/c4t6d0DDS
/dev/rmt/0mb /dev/rmt/c4t6d0BESTb /dev/rmt/c4t6d0DDSb
/dev/rmt/0mn /dev/rmt/c4t6d0BESTn /dev/rmt/c4t6d0DDSn
/dev/rmt/0mnb /dev/rmt/c4t6d0BESTnb /dev/rmt/c4t6d0DDSnb
target 12 0/4/0/0.7 tgt CLAIMED DEVICE
ctl 5 0/4/0/0.7.0 sctl CLAIMED DEVICE Initiator
/dev/rscsi/c4t7d0
target 13 0/4/0/0.8 tgt CLAIMED DEVICE
disk 6 0/4/0/0.8.0 sdisk CLAIMED DEVICE SEAGATE ST318203LC
/dev/dsk/c4t8d0 /dev/rdsk/c4t8d0
target 14 0/4/0/0.9 tgt CLAIMED DEVICE
disk 7 0/4/0/0.9.0 sdisk CLAIMED DEVICE SEAGATE ST118202LC
/dev/dsk/c4t9d0 /dev/rdsk/c4t9d0
target 15 0/4/0/0.10 tgt CLAIMED DEVICE
disk 8 0/4/0/0.10.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c4t10d0 /dev/rdsk/c4t10d0
target 16 0/4/0/0.11 tgt CLAIMED DEVICE
disk 9 0/4/0/0.11.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c4t11d0 /dev/rdsk/c4t11d0
target 17 0/4/0/0.12 tgt CLAIMED DEVICE
disk 10 0/4/0/0.12.0 sdisk CLAIMED DEVICE SEAGATE ST318404LC
/dev/dsk/c4t12d0 /dev/rdsk/c4t12d0
target 18 0/4/0/0.14 tgt CLAIMED DEVICE
ctl 6 0/4/0/0.14.0 sctl CLAIMED DEVICE HP A5272A
/dev/rscsi/c4t14d0
ba 5 0/5 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
ba 6 0/6 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
ba 7 0/7 lba CLAIMED BUS_NEXUS Local PCI Bus Adapter (782)
lan 1 0/7/0/0 btlan5 CLAIMED INTERFACE HP A5230A/B5509BA PCI 10/100Base-TX
Addon
/dev/diag/lan1 /dev/ether1 /dev/lan1
memory 0 8 memory CLAIMED MEMORY Memory
processor 0 160 processor CLAIMED PROCESSOR Processor
processor 1 162 processor CLAIMED PROCESSOR Processor
processor 2 166 processor CLAIMED PROCESSOR Processor


Can anyone give me any pointers as to what the issue may be please ?

Does it look like a disk issue or worse than that ??

:o(

TIA
Sean

[Muthukumar_5]

when ioscan software state is showing NO_HW then hardware in that address not detectable.

NO_HW the hardware at this address is no
longer responding

~regards

[Arunvijai_4]

Dont be afraid, take a look at these threads for urgent help

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=798502
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=842933
http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=837456

and

http://www1.itrc.hp.com/service/james/search.do?todo=search&admit=-682735245+1131363386958+28353475&searchtext=NO_HW&from=forums&origin=0&submit.x=0&wpa=forums1.itrc.hp.com%3A80&submit.y=0&searchcategory=ALL&hpl=1&searchcriteria=allwords&rn=25&source=7000&presort=rank&chkServStor=on&esc=us.support.itrc.hp.com

-Arun

[Andy Miller_4]

I think the issue is getting a little worse, as the following have just occured also........

Nov 7 11:22:51 sul7it11 vmunix: LVM: VG 64 0x020000: PVLink 31 0x009000 Failed! The PV is not accessible.
Nov 7 11:22:53 sul7it11 EMS [4018]: ------ EMS Event Notification ------ Value: "CRITICAL (5)" for Resource: "/storage/events/disks/default/0_0_1_0.8.0" (Threshold: >= " 3") Execute the following command to obtain event details: /opt/resmon/bin/resdata -R 263323650 -r /storage/events/disks/default/0_0_1_0.8.0 -n 263323680 -a
Nov 7 11:22:54 sul7it11 EMS [4018]: ------ EMS Event Notification ------ Value: "CRITICAL (5)" for Resource: "/storage/events/disks/default/0_0_1_0.9.0" (Threshold: >= " 3") Execute the following command to obtain event details: /opt/resmon/bin/resdata -R 263323653 -r /storage/events/disks/default/0_0_1_0.9.0 -n 263323681 -a
Nov 7 11:22:57 sul7it11 vmunix: LVM: VG 64 0x020000: PVLink 31 0x009000 Recovered.
Nov 7 11:22:58 sul7it11 vmunix: LVM: VG 64 0x020000: PVLink 31 0x00b000 Recovered.
Nov 7 11:24:21 sul7it11 vmunix: SCSI: Abort abandoned -- lbolt: 266442873, dev: 1f00b000, io_id: ab5d81, status: 200
Nov 7 11:24:22 sul7it11 vmunix:
Nov 7 11:24:22 sul7it11 vmunix: SCSI: Read error -- dev: b 31 0x00b000, errno: 126, resid: 1024,
Nov 7 11:24:22 sul7it11 vmunix: blkno: 8, sectno: 16, offset: 8192, bcount: 1024.
Nov 7 11:24:22 sul7it11 vmunix: LVM: VG 64 0x020000: PVLink 31 0x00b000 Failed! The PV is not accessible.




# /events/disks/default/0_0_1_0.9.0 -n 263323681 -a <

CURRENT MONITOR DATA:

Event Time..........: Mon Nov 7 11:22:53 2005
Severity............: CRITICAL
Monitor.............: disk_em
Event #.............: 17
System..............: sul7it11

Summary:
Disk at hardware path 0/0/1/0.9.0 : I/O request failed.


Description of Error:

The hardware responded to the initial query, but then it stopped
responding to more requests by the driver. The I/O request was not
completed.

Probable Cause / Recommended Action:

The bus device reset may have occurred or the device has failed. The bus
device reset could have occurred because the power was cycled or that the
device is a part of an enclosure and a device in that enclosure was pulled
out or put in, or that the interface card had a problem and it reset the
bus. If these errors continue, there may be a problem with the device or
the card.

Additional Event Data:
System IP Address...: 127.127.72.192
System IP Address...: 127.127.72.192
Event Id............: 0x436f390d00000000
Monitor Version.....: B.01.00
Event Class.........: I/O
Client Configuration File...........:
/var/stm/config/tools/monitor/default_disk_em.clcfg
Client Configuration File Version...: A.01.00
Qualification criteria met.
Number of events..: 1
Associated OS error log entry id(s):
0x436f390600000002
Additional System Data:
System Model Number.............: 9000/800/L2000-44
OS Version......................: B.11.00
STM Version.....................: A.42.00
EMS Version.....................: A.03.20
Latest information on this event:
http://docs.hp.com/hpux/content/hardware/ems/disk_em.htm#17

v-v-v-v-v-v-v-v-v-v-v-v-v D E T A I L S v-v-v-v-v-v-v-v-v-v-v-v-v



Component Data:
Physical Device Path...: 0/0/1/0.9.0
Device Class...........: Disk
Inquiry Vendor ID......: SEAGATE
Inquiry Product ID.....: ST318404LC
Firmware Version.......: HP01
Serial Number..........: 3BT0524E000010411Y03

Product/Device Identification Information:

Logger ID.........: sdisk
Product Identifier: SCSI Disk
Product Qualifier.: SEAGATEST318404LC
SCSI Target ID....: 0x09
SCSI LUN..........: 0x00

I/O Log Event Data:

Driver Status Code..................: 0x0000007E
Length of Logged Hardware Status....: 4 bytes.
Offset to Logged Manager Information: 8 bytes.
Length of Logged Manager Information: 34 bytes.

SCSI Command Data Block:

Command Data Block Contents:
0x0000: 28 00 00 A7 CB A0 00 00 10 00

Command Data Block Fields (10-byte fmt):
Command Operation Code...(0x28)..: READ
Logical Unit Number..............: 0
DPO Bit..........................: 0
FUA Bit..........................: 0
Relative Address Bit.............: 0
Logical Block Address............: 10996640 (0x00A7CBA0)
Transfer Length..................: 16 (0x0010)

Manager-Specific Data Fields:
Request ID.............: 0x00AB1FBC
Data Residue...........: 0x00002000
CDB status.............: 0x00000400
Sense Status...........: 0x00000000
Bus ID.................: 0x00
Target ID..............: 0x09
LUN ID.................: 0x00
Sense Data Length......: 0x00
Q Tag..................: 0x72
Retry Count............: 1




# /events/disks/default/0_0_1_0.8.0 -n 263323680 -a <

CURRENT MONITOR DATA:

Event Time..........: Mon Nov 7 11:22:52 2005
Severity............: CRITICAL
Monitor.............: disk_em
Event #.............: 17
System..............: sul7it11

Summary:
Disk at hardware path 0/0/1/0.8.0 : I/O request failed.


Description of Error:

The hardware responded to the initial query, but then it stopped
responding to more requests by the driver. The I/O request was not
completed.

Probable Cause / Recommended Action:

The bus device reset may have occurred or the device has failed. The bus
device reset could have occurred because the power was cycled or that the
device is a part of an enclosure and a device in that enclosure was pulled
out or put in, or that the interface card had a problem and it reset the
bus. If these errors continue, there may be a problem with the device or
the card.

Additional Event Data:
System IP Address...: 127.127.72.192
System IP Address...: 127.127.72.192
Event Id............: 0x436f390c00000000
Monitor Version.....: B.01.00
Event Class.........: I/O
Client Configuration File...........:
/var/stm/config/tools/monitor/default_disk_em.clcfg
Client Configuration File Version...: A.01.00
Qualification criteria met.
Number of events..: 1
Associated OS error log entry id(s):
0x436f390600000001
Additional System Data:
System Model Number.............: 9000/800/L2000-44
OS Version......................: B.11.00
STM Version.....................: A.42.00
EMS Version.....................: A.03.20
Latest information on this event:
http://docs.hp.com/hpux/content/hardware/ems/disk_em.htm#17

v-v-v-v-v-v-v-v-v-v-v-v-v D E T A I L S v-v-v-v-v-v-v-v-v-v-v-v-v



Component Data:
Physical Device Path...: 0/0/1/0.8.0
Device Class...........: Disk
Inquiry Vendor ID......: SEAGATE
Inquiry Product ID.....: ST318404LC
Firmware Version.......: HP04
Serial Number..........: 3BT1SA0L00002116JC24

Product/Device Identification Information:

Logger ID.........: sdisk
Product Identifier: SCSI Disk
Product Qualifier.: SEAGATEST318404LC
SCSI Target ID....: 0x08
SCSI LUN..........: 0x00

I/O Log Event Data:

Driver Status Code..................: 0x0000007E
Length of Logged Hardware Status....: 4 bytes.
Offset to Logged Manager Information: 8 bytes.
Length of Logged Manager Information: 34 bytes.

SCSI Command Data Block:

Command Data Block Contents:
0x0000: 2A 00 00 7A 18 E4 00 00 02 00

Command Data Block Fields (10-byte fmt):
Command Operation Code...(0x2A)..: WRITE
Logical Unit Number..............: 0
DPO Bit..........................: 0
FUA Bit..........................: 0
Relative Address Bit.............: 0
Logical Block Address............: 8001764 (0x007A18E4)
Transfer Length..................: 2 (0x0002)

Manager-Specific Data Fields:
Request ID.............: 0x00AB1FC3
Data Residue...........: 0x00000400
CDB status.............: 0x00000400
Sense Status...........: 0x00000000
Bus ID.................: 0x00
Target ID..............: 0x08
LUN ID.................: 0x00
Sense Data Length......: 0x00
Q Tag..................: 0x78
Retry Count............: 1



The HP link http://docs.hp.com/hpux/content/hardware/ems/disk_em.htm#17 details the following...........

Event 17

Severity: Critical
Event Summary: I/O request failed.
Event Class: I/O
Problem Description: The hardware responded to the initial query, but then it stopped responding to more requests by the driver. The I/O request was not completed.
Probable Cause / Recommended Action: The bus device reset may have occurred or the device has failed. The bus device reset could have occurred because the power was cycled or that the device is a part of an enclosure and a device in that enclosure was pulled out or put in, or that the interface card had a problem and it reset thebus. If these errors continue, there may be a problem with the device or the card.
Automated Recovery: None.
Event Generation Threshold: 1 occurrence
Fault Notifier Generation Threshold: 1 occurrence

Does anyone think the BUS is on the way out ???

[Stephen Keane]

Make sure the SCSI cabling is OK, especially whether any plugs are loose or not fully inserted.

[Andy Miller_4]

Yes, all cables are securely seated.

we do have a tape drive 'daisy chained' but it has been like this for a while, and is not on one of the addresses showing as having a problem............