HP-UX 11.11 not seeing EVA3000 disks over replaced FC adapter

[Niksa Franceschi]

Hi,

HP-UX server is rp7420 with HP-UX 11.11, and storage is EVA3000.
Originally HP-UX was connected via FC switch to EVA3000, but we moved it to our SAN infrastructure. During this maintenance it turned out one of our FC adapters on HP was defective.
After move to SAN infrastructure, I was able to see EVA disks over working port (no need to change anything on host or EVA).
After this we had maintenance of replacing failed adapter.
Server was booted, EVA could see new WWN of new FC adapter. After adding on EVA new Port with WWN corresponding to new FC adapter, it is still not possible to see new disks on HP-UX server.

Adapter in question is /dev/td0

Here are some outputs, which puzzle me a bit:
# fcmsutil /dev/td0

Vendor ID is = 0x00103c
Device ID is = 0x001029
XL2 Chip Revision No is = 2.3
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x00128c
Topology = PTTOPT_FABRIC
Link Speed = 2Gb
Local N_Port_id is = 0x342700
N_Port Node World Wide Name = 0x50060b000023708b
N_Port Port World Wide Name = 0x50060b000023708a
Driver state = ONLINE
Hardware Path is = 1/0/2/1/0
Number of Assisted IOs = 5261
Number of Active Login Sessions = 0
Dino Present on Card = NO
Maximum Frame Size = 2048
Driver Version = @(#) libtd.a HP Fibre Channel Tachyon TL/TS/XL2 Driver B.11.11.12 (AR1204) /ux/kern/kisu/TL/src/common/wsio/td_glue.c: Oct 11 2004, 14:45:41

# fcmsutil /dev/td1

Vendor ID is = 0x00103c
Device ID is = 0x001029
XL2 Chip Revision No is = 2.3
PCI Sub-system Vendor ID is = 0x00103c
PCI Sub-system ID is = 0x00128c
Topology = PTTOPT_FABRIC
Link Speed = 2Gb
Local N_Port_id is = 0x332700
N_Port Node World Wide Name = 0x50060b00002cee47
N_Port Port World Wide Name = 0x50060b00002cee46
Driver state = ONLINE
Hardware Path is = 1/0/4/1/0
Number of Assisted IOs = 32499082
Number of Active Login Sessions = 1
Dino Present on Card = NO
Maximum Frame Size = 2048
Driver Version = @(#) libtd.a HP Fibre Channel Tachyon TL/TS/XL2 Driver B.11.11.12 (AR1204) /ux/kern/kisu/TL/src/common/wsio/td_glue.c: Oct 11 2004, 14:45:41

# fcmsutil /dev/td0 get remote all

Target N_Port_id is = 0x342800
Target state = DVS_UNOPENED
Symbolic Name = HSV100 - EVA - CR04BB
Port Type = N_PORT
SLER Capable (supports Retry & TRID) = NO
Target Port World Wide Name = 0x50001fe150023099
Target Node World Wide Name = 0x50001fe150023090

Common Service parameters (all values shown in hex):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Version_Hi: 9 Version_Low: 9
BB_Credit: 0 Ciro: 1
Alt_BBcred: 1 Rxsz: 800
Conseq: ff RO_Bitmap: 1
Resolution: 0 EDTOV(ms): 0

Vendor version (in hex) 0 0 0 0
-------------------------------------------------------------

# fcmsutil /dev/td1 get remote all

Target N_Port_id is = 0x332800
Target state = DVS_READY
Symbolic Name = HSV100 - EVA - CR04BB
Port Type = N_PORT
SLER Capable (supports Retry & TRID) = NO
Target Port World Wide Name = 0x50001fe15002309d
Target Node World Wide Name = 0x50001fe150023090

Common Service parameters (all values shown in hex):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Version_Hi: 9 Version_Low: 9
BB_Credit: 0 Ciro: 1
Alt_BBcred: 1 Rxsz: 800
Conseq: ff RO_Bitmap: 1
Resolution: 0 EDTOV(ms): 0

Vendor version (in hex) 0 0 0 0
-------------------------------------------------------------

# spmgr display
Server: hpprojx.vipnet.hr Report Created: Wed, Apr 29 10:19:14 2009
Command: spmgr display
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Storage: 5000-1FE1-5002-3090
Load Balance: Off Auto-restore: Off
Path Verify: On Verify Interval: 30
HBAs: td1
Controller: P66C5D4AAP106B, Operational
Devices: c12t0d0 c12t0d3 c12t0d4

TGT/LUN Device WWLUN_ID H/W_Path #_Paths
0/ 0 c12t0d0 6005-08B4-0001-1A5F-0001-3000-008B-0000 1
255/255/0/0.0
Controller Path_Instance HBA Preferred? Path_Status
P66C5D4AAP106B no
c13t0d1 td1 no Active


TGT/LUN Device WWLUN_ID H/W_Path #_Paths
0/ 3 c12t0d3 6005-08B4-0001-15BF-0001-2000-0085-0000 1
255/255/0/0.3
Controller Path_Instance HBA Preferred? Path_Status
P66C5D4AAP106B no
c13t0d2 td1 no Active


TGT/LUN Device WWLUN_ID H/W_Path #_Paths
0/ 4 c12t0d4 6005-08B4-0001-15BF-0001-2000-008B-0000 1
255/255/0/0.4
Controller Path_Instance HBA Preferred? Path_Status
P66C5D4AAP106B no
c13t0d3 td1 no Active


# ioscan -fnkH 1/0/2/1/0
Class I H/W Path Driver S/W State H/W Type Description
=========================================================================
fc 0 1/0/2/1/0 td CLAIMED INTERFACE HP Tachyon XL2 Fibre Channel Mass Storage Adapter
/dev/td0
fcp 3 1/0/2/1/0.52 fcp CLAIMED INTERFACE FCP Domain
ext_bus 15 1/0/2/1/0.52.40.0.0 fcparray CLAIMED INTERFACE FCP Array Interface
target 7 1/0/2/1/0.52.40.0.0.0 tgt CLAIMED DEVICE
ctl 74 1/0/2/1/0.52.40.0.0.0.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t0d0
target 8 1/0/2/1/0.52.40.0.0.1 tgt CLAIMED DEVICE
ctl 75 1/0/2/1/0.52.40.0.0.1.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t1d0
target 9 1/0/2/1/0.52.40.0.0.2 tgt CLAIMED DEVICE
ctl 76 1/0/2/1/0.52.40.0.0.2.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t2d0
target 10 1/0/2/1/0.52.40.0.0.3 tgt CLAIMED DEVICE
ctl 77 1/0/2/1/0.52.40.0.0.3.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t3d0
target 11 1/0/2/1/0.52.40.0.0.4 tgt CLAIMED DEVICE
ctl 78 1/0/2/1/0.52.40.0.0.4.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t4d0
target 12 1/0/2/1/0.52.40.0.0.5 tgt CLAIMED DEVICE
ctl 79 1/0/2/1/0.52.40.0.0.5.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t5d0
target 13 1/0/2/1/0.52.40.0.0.6 tgt CLAIMED DEVICE
ctl 80 1/0/2/1/0.52.40.0.0.6.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t6d0
target 14 1/0/2/1/0.52.40.0.0.7 tgt CLAIMED DEVICE
ctl 81 1/0/2/1/0.52.40.0.0.7.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t7d0
target 15 1/0/2/1/0.52.40.0.0.8 tgt CLAIMED DEVICE
ctl 82 1/0/2/1/0.52.40.0.0.8.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t8d0
target 16 1/0/2/1/0.52.40.0.0.9 tgt CLAIMED DEVICE
ctl 83 1/0/2/1/0.52.40.0.0.9.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t9d0
target 17 1/0/2/1/0.52.40.0.0.10 tgt CLAIMED DEVICE
ctl 84 1/0/2/1/0.52.40.0.0.10.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t10d0
target 18 1/0/2/1/0.52.40.0.0.11 tgt CLAIMED DEVICE
ctl 85 1/0/2/1/0.52.40.0.0.11.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t11d0
target 19 1/0/2/1/0.52.40.0.0.12 tgt CLAIMED DEVICE
ctl 86 1/0/2/1/0.52.40.0.0.12.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t12d0
target 20 1/0/2/1/0.52.40.0.0.13 tgt CLAIMED DEVICE
ctl 87 1/0/2/1/0.52.40.0.0.13.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t13d0
target 21 1/0/2/1/0.52.40.0.0.14 tgt CLAIMED DEVICE
ctl 88 1/0/2/1/0.52.40.0.0.14.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t14d0
target 22 1/0/2/1/0.52.40.0.0.15 tgt CLAIMED DEVICE
ctl 89 1/0/2/1/0.52.40.0.0.15.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c15t15d0
ext_bus 16 1/0/2/1/0.52.40.255.0 fcpdev CLAIMED INTERFACE FCP Device Interface
target 23 1/0/2/1/0.52.40.255.0.0 tgt CLAIMED DEVICE
ctl 90 1/0/2/1/0.52.40.255.0.0.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c16t0d0
# ioscan -fnkH 1/0/4/1/0
Class I H/W Path Driver S/W State H/W Type Description
=========================================================================
fc 1 1/0/4/1/0 td CLAIMED INTERFACE HP Tachyon XL2 Fibre Channel Mass Storage Adapter
/dev/td1
fcp 2 1/0/4/1/0.51 fcp CLAIMED INTERFACE FCP Domain
ext_bus 13 1/0/4/1/0.51.40.0.0 fcparray CLAIMED INTERFACE FCP Array Interface
target 24 1/0/4/1/0.51.40.0.0.0 tgt CLAIMED DEVICE
ctl 72 1/0/4/1/0.51.40.0.0.0.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c13t0d0
hsx 12 1/0/4/1/0.51.40.0.0.0.1 hsx CLAIMED DEVICE HP HSV100
hsx 13 1/0/4/1/0.51.40.0.0.0.2 hsx CLAIMED DEVICE HP HSV100
hsx 14 1/0/4/1/0.51.40.0.0.0.3 hsx CLAIMED DEVICE HP HSV100
ext_bus 14 1/0/4/1/0.51.40.255.0 fcpdev CLAIMED INTERFACE FCP Device Interface
target 25 1/0/4/1/0.51.40.255.0.0 tgt CLAIMED DEVICE
ctl 73 1/0/4/1/0.51.40.255.0.0.0 sctl CLAIMED DEVICE HP HSV100
/dev/rscsi/c14t0d0
#


Interesting part is output of: fcmsutil /dev/td0 get remote all, where it says: Target state = DVS_UNOPENED
Any ideas what could be the problem and how to fix it, so I can see disks over /dev/td0 too?

[IBaltay]

Hi,
did you allready changed the:
a) PWWNs in the EVA host groups
b) if you are using the PWWN zoning (soft zoning), have you changed the PWWNs in the zones
?

[Niksa Franceschi]

Hi,

yes I added new Port WWN on EVA side (under Host Properties -> Ports) for new adapter.
As for zones, they are based on port-zoning on HP Directors. Our SAN guy double-checked configuration on both ports (one working and this one which is not working), and all is correct. Also, he tried to 'refresh' them just in case, yet still the same.
Interesting is that both sides can see remote WWN, yet I cannot see disks, and difference in output of ioscan for each adapter.

I must say I did not try to reboot HP server after I added new WWN on EVA side, and if possible would prefer to avoid it, as we have database running on server (yes, I tried several ioscan's, insf's - with and without -e option).

[IBaltay]

you need to run ioscan command:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1096460

and fcmsutil command:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=193929

[Niksa Franceschi]

I've tried several times with ioscan command.
I now tried also command: fcmsutil /dev/td0 replace_dsk 0x342800, but again to no avail.

I also added output of other fcmsutil command into attached document.
What bothers me is output of 'fcmsutil /dev/td0 get remote all' output where is still says:
Target state = DVS_UNOPENED
compared to: 'Target state = DVS_READY' on adapter where it's working.
If you have any more ideas, please let me know!

[IBaltay]

Hi,
maybe here could be some ideas:
http://sysdoc.doors.ch/HP/lpg29058.pdf

[IBaltay]

or this could be related:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=739802

[IBaltay]

this abstract could be interesting:
DVS_READY means that the storage device allow initiator logins on the port; DVS_UNOPENED may appear in cases where the WWN of the hba is not registered on a Clariion Array or when a DMX storage port (VCM Enabled) is accesible but no device was masked for that particular port or even in cases where a storage port was moved in SAN from one port-id to another so the host does not authenticate it anymore.
Generally, it means that something is happening with that device but this may only be interpreted on an active device: if the host does not try to acces luns on that specific storage it may be possible that the state of the device to be unopened.
I guess that this state means that the host did not tried to open (login in fabric terms) the devi ce in the last period of time (AFAIK the hp-ux logins to the device each time it opens it) or tried and failed.