FCP NO_HW 문제

[김용경]

#ioscan -f



Class I H/W Path Driver S/W State H/W Type Description

====================================================================================

root 0 root CLAIMED BUS_NEXUS

ioa 0 0 sba CLAIMED BUS_NEXUS System Bus Adapter (4030)

ba 0 0/0 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (122e)

tty 0 0/0/1/0 rmp3f01 CLAIMED INTERFACE PCI class(255,0) (103c1303)

tty 1 0/0/1/1 rmp3f01 CLAIMED INTERFACE PCI SimpleComm (103c1302)

tty 2 0/0/1/2 asio0 CLAIMED INTERFACE PCI Serial (103c1048)

OO 0 0/0/2/0 UsbOhci CLAIMED INTERFACE USB OHCI Interface

OO 1 0/0/2/1 UsbOhci CLAIMED INTERFACE USB OHCI Interface

OO 2 0/0/2/1.0 UsbMiniBus CLAIMED INTERFACE USB Composite Device

OO 3 0/0/2/1.0.0 UsbBulkOnlyMS CLAIMED DEVICE USB Bulk Only

ext_bus 0 0/0/2/1.0.16 UsbScsiAdaptor CLAIMED INTERFACE USB SCSI Stack Adaptor

target 0 0/0/2/1.0.16.0 tgt CLAIMED DEVICE

disk 0 0/0/2/1.0.16.0.0 sdisk CLAIMED DEVICE TEAC DV-28E-V

target 1 0/0/2/1.0.16.7 tgt CLAIMED DEVICE

ctl 0 0/0/2/1.0.16.7.0 sctl CLAIMED DEVICE Initiator

OO 4 0/0/2/2 UsbEhci CLAIMED INTERFACE USB EHCI Interface

ba 1 0/1 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (122e)

graphics 0 0/1/1/0 gvid_core CLAIMED INTERFACE PCI Display (10025157)

ba 2 0/2 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (122e)

ba 3 0/3 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (12ee)

ba 4 0/4 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (122e)

escsi_ctlr 0 0/4/1/0 sasd CLAIMED INTERFACE HP PCI/PCI-X SAS MPT Adapter

ext_bus 1 0/4/1/0.0.0 sasd_vbus CLAIMED INTERFACE SAS Device Interface

target 2 0/4/1/0.0.0.0 tgt CLAIMED DEVICE

disk 1 0/4/1/0.0.0.0.0 sdisk CLAIMED DEVICE HP IR Volume

lan 0 0/4/2/0 iether CLAIMED INTERFACE HP AB352-60003 PCI/PCI-X 1000Base-T Dual-port Core

lan 1 0/4/2/1 iether CLAIMED INTERFACE HP AB352-60003 PCI/PCI-X 1000Base-T Dual-port Core

ba 5 0/5 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (122e)

ba 6 0/6 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (122e)

ba 7 0/7 lba CLAIMED BUS_NEXUS Local PCI-X Bus Adapter (12ee)

fc 0 0/7/1/0 fcd CLAIMED INTERFACE HP AB379-60101 4Gb Dual Port PCI/PCI-X Fibre Channel Adapter (FC

Port 1)

fcp 0 0/7/1/0.1 fcd_fcp CLAIMED INTERFACE FCP Domain

ext_bus 7 0/7/1/0.1.0.0.0 fcd_vbus NO_HW INTERFACE FCP Array Interface

target 6 0/7/1/0.1.0.0.0.0 tgt NO_HW DEVICE

disk 3 0/7/1/0.1.0.0.0.0.0 sdisk NO_HW DEVICE NETAPP LUN

ext_bus 2 0/7/1/0.1.0.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface

target 5 0/7/1/0.1.0.255.0.0 tgt CLAIMED DEVICE

ctl 2 0/7/1/0.1.0.255.0.0.0 sctl CLAIMED DEVICE NETAPP LUN

ext_bus 3 0/7/1/0.1.6.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface

fc 1 0/7/1/1 fcd CLAIMED INTERFACE HP AB379-60101 4Gb Dual Port PCI/PCI-X Fibre Channel Adapter (FC

Port 2)

fcp 1 0/7/1/1.1 fcd_fcp CLAIMED INTERFACE FCP Domain

ext_bus 6 0/7/1/1.1.0.0.0 fcd_vbus NO_HW INTERFACE FCP Array Interface

target 4 0/7/1/1.1.0.0.0.0 tgt NO_HW DEVICE

disk 2 0/7/1/1.1.0.0.0.0.0 sdisk NO_HW DEVICE NETAPP LUN

ext_bus 4 0/7/1/1.1.0.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface

target 3 0/7/1/1.1.0.255.0.0 tgt CLAIMED DEVICE

ctl 1 0/7/1/1.1.0.255.0.0.0 sctl CLAIMED DEVICE NETAPP LUN

ext_bus 5 0/7/1/1.1.6.255.0 fcd_vbus CLAIMED INTERFACE FCP Device Interface

processor 0 120 processor CLAIMED PROCESSOR Processor

processor 1 121 processor CLAIMED PROCESSOR Processor

processor 2 122 processor CLAIMED PROCESSOR Processor

processor 3 123 processor CLAIMED PROCESSOR Processor

ba 8 250 pdh CLAIMED BUS_NEXUS Core I/O Adapter

ipmi 0 250/0 ipmi CLAIMED INTERFACE IPMI Controller

tty 3 250/1 asio0 CLAIMED INTERFACE Built-in RS232C

acpi_node 0 250/2 acpi_node CLAIMED INTERFACE Acpi Hardware



----------------------------------------

#ioscan -funCfc



Class I H/W Path Driver S/W State H/W Type Description

=================================================================

fc 0 0/7/1/0 fcd CLAIMED INTERFACE HP AB379-60101 4Gb Dual Port PCI/PCI-X Fibre Channel Adapter (FC Port 1)

/dev/fcd0

fc 1 0/7/1/1 fcd CLAIMED INTERFACE HP AB379-60101 4Gb Dual Port PCI/PCI-X Fibre Channel Adapter (FC Port 2)

/dev/fcd1



-------------------------------------------------------

#fcmsutil /dev/fcd0



Vendor ID is = 0x001077

Device ID is = 0x002422

PCI Sub-system Vendor ID is = 0x00103c

PCI Sub-system ID is = 0x0012d7

PCI Mode = PCI-X 266 MHz

ISP Code version = 4.0.91

ISP Chip version = 3

Topology = PTTOPT_FABRIC

Link Speed = 2Gb

Local N_Port_id is = 0x010700

Previous N_Port_id is = 0x010700

N_Port Node World Wide Name = 0x5001438001326e9d

N_Port Port World Wide Name = 0x5001438001326e9c

Switch Port World Wide Name = 0x200700051e0d6afd

Switch Node World Wide Name = 0x100000051e0d6afd

Driver state = ONLINE

Hardware Path is = 0/7/1/0

Maximum Frame Size = 2048

Driver-Firmware Dump Available = YES

Driver-Firmware Dump Timestamp = Tue Jan 20 22:34:27 2009

Driver Version = @(#) libfcd.a HP Fibre Channel ISP 23xx & 24xx Driver B.11.23.08.02 /ux/core/isu/FCD/kern/src/common/wsio/fcd_init.c:Sep 24 2007,00:39:23







#fcmsutil /dev/fcd1



Vendor ID is = 0x001077

Device ID is = 0x002422

PCI Sub-system Vendor ID is = 0x00103c

PCI Sub-system ID is = 0x0012d7

PCI Mode = PCI-X 266 MHz

ISP Code version = 4.0.91

ISP Chip version = 3

Topology = PTTOPT_FABRIC

Link Speed = 2Gb

Local N_Port_id is = 0x010700

Previous N_Port_id is = 0x010700

N_Port Node World Wide Name = 0x5001438001326e9f

N_Port Port World Wide Name = 0x5001438001326e9e

Switch Port World Wide Name = 0x200700051e0d62db

Switch Node World Wide Name = 0x100000051e0d62db

Driver state = ONLINE

Hardware Path is = 0/7/1/1

Maximum Frame Size = 2048

Driver-Firmware Dump Available = NO

Driver-Firmware Dump Timestamp = N/A

Driver Version = @(#) libfcd.a HP Fibre Channel ISP 23xx & 24xx Driver B.11.23.08.02 /ux/core/isu/FCD/kern/src/common/wsio/fcd_init.c:Sep 24 2007,00:39:23

--------------------------------------------------------



#fcdlist



/dev/fcd0 0/7/1/0.1.0.255.0.0.0 fcd_vbus 0x010000

CLAIMED DEVICE NETAPP LUN /dev/rscsi/c2t0d0



/dev/fcd0 0/7/1/0.1.0.0.0 fcd_vbus 0x010000

NO_HW DEVICE NETAPP LUN /dev/dsk/c7t0d0 /dev/rdsk/c7t0d0



/dev/fcd1 0/7/1/1.1.0.255.0.0.0 fcd_vbus 0x010000

CLAIMED DEVICE NETAPP LUN /dev/rscsi/c4t0d0



/dev/fcd1 0/7/1/1.1.0.0.0 fcd_vbus 0x010000

NO_HW DEVICE NETAPP LUN /dev/dsk/c6t0d0 /dev/rdsk/c6t0d0





---------------------------------------------





#bdf

/dev/vg01/lvol1 20480000 8082786 11622398 41% /app1

/dev/vg01/lvol2 31936512 29079553 2678426 92% /data



정상적으로 서비스 중이며 failover test에도 이상이 없음.



--------------------------------------------------------

#ioscan -funCdisk

Class I H/W Path Driver S/W State H/W Type Description

====================================================================================

disk 0 0/0/2/1.0.16.0.0 sdisk CLAIMED DEVICE TEAC DV-28E-V

/dev/dsk/c0t0d0 /dev/rdsk/c0t0d0

disk 1 0/4/1/0.0.0.0.0 sdisk CLAIMED DEVICE HP IR Volume

/dev/dsk/c1t0d0 /dev/dsk/c1t0d0s2 /dev/rdsk/c1t0d0 /dev/rdsk/c1t0d0s2

/dev/dsk/c1t0d0s1 /dev/dsk/c1t0d0s3 /dev/rdsk/c1t0d0s1 /dev/rdsk/c1t0d0s3

disk 3 0/7/1/0.1.0.0.0.0.0 sdisk NO_HW DEVICE NETAPP LUN

/dev/dsk/c7t0d0 /dev/rdsk/c7t0d0

disk 2 0/7/1/1.1.0.0.0.0.0 sdisk NO_HW DEVICE NETAPP LUN

/dev/dsk/c6t0d0 /dev/rdsk/c6t0d0

----------------------------------------------------------



정기점검시 발생된 문제를 발견하였지만 해결 방법을 찾지 못하여 도움을 구합니다.



storage san path 정상

san switch 정상

hpux 정상

fcms util 정상



ioscan -f 시 NO_HW 확인

ioscan -funC NO_HW 확인



bdf시 san 의 2개의 path 로 연결된 볼륨이 전부 읽고 쓰기가 가능

failover test시에도 정상적으로 작동



현재 상태에서 san을 통하여 lun 을 한개 더 생성 하게되면 스토리지에선 제대로 뿌려주나 hpux쪽에서 찾지를못합니다.

NO_HW 원인을 찾을 방법이 없을까요?

[김병수]

san에 연결된 disk array가 NetApp 인가요..



모든것이 정상이라면 san 환경에서는 san switch의 firmware를 upgrade하여 보실것을 권장합니다.



그럼~~~

[고재진]

안녕하세요.



모든것이 정상이라면 케이블을 뺐다가 다시 꽂아보세요.

제대로 sync를 못해서 그럴수도 있습니다.



그럼.