HP-UX 11.31 Multipathing w/EMC Clarion

[Scott Giddens]

I am told the native multipathing in 11iv3 works with EMC FC storage but does anyone know the trick to turn it on? The Storage Group at work provided me with two paths to a LUN on a CX3-80 but four show up.

I loaded the latest scsimgr patch and was finaly able to see the status flag for the Asymmetric States. It says N/A support for asymmetric states and shows 2 of the four LUNpaths as DISABLE_OPENS which there is no discription as to what that means in the documentation. here are the results of a LUN query:

scsimgr get_info all_lpt -D /dev/rdisk/disk12

STATUS INFORMATION FOR LUN PATH : lunpath8

Generic Status Information

SCSI services internal state = UNOPEN
Open close state = ACTIVE
Protocol = fibre_channel
root@NASTRAN7> scsimgr get_info all_lpt -D /dev/rdisk/disk12

STATUS INFORMATION FOR LUN PATH : lunpath8

Generic Status Information

SCSI services internal state = UNOPEN
Open close state = ACTIVE
Protocol = fibre_channel
EVPD page 0x83 description code = 1
EVPD page 0x83 description association = 0
EVPD page 0x83 description type = 3
World Wide Identifier (WWID) = 0x6006016000e71e009c4e2eb7a794dd11
Total number of Outstanding I/Os = 0
Maximum I/O timeout in seconds = 30
Maximum I/O size allowed = 2097152
Maximum number of active I/Os allowed = 8
Maximum queue depth = 8
Queue full delay count = 0
Asymmetric state = NA
Device preferred path = No
Relative target port identifier = 5
Target port group identifier = 2

STATUS INFORMATION FOR LUN PATH : lunpath9

Generic Status Information

SCSI services internal state = UNOPEN
Open close state = DISABLE_OPENS
Protocol = fibre_channel
EVPD page 0x83 description code = 1
EVPD page 0x83 description association = 0
EVPD page 0x83 description type = 3
World Wide Identifier (WWID) = 0x6006016000e71e009c4e2eb7a794dd11
Total number of Outstanding I/Os = 0
Maximum I/O timeout in seconds = 30
Maximum I/O size allowed = 2097152
Maximum number of active I/Os allowed = 8
Maximum queue depth = 8
Queue full delay count = 0
Asymmetric state = NA
Device preferred path = No
Relative target port identifier = 2
Target port group identifier = 1

STATUS INFORMATION FOR LUN PATH : lunpath10

Generic Status Information

SCSI services internal state = UNOPEN
Open close state = ACTIVE
Protocol = fibre_channel
EVPD page 0x83 description code = 1
EVPD page 0x83 description association = 0
EVPD page 0x83 description type = 3
World Wide Identifier (WWID) = 0x6006016000e71e009c4e2eb7a794dd11
Total number of Outstanding I/Os = 0
Maximum I/O timeout in seconds = 30
Maximum I/O size allowed = 2097152
Maximum number of active I/Os allowed = 8
Maximum queue depth = 8
Queue full delay count = 0
Asymmetric state = NA
Device preferred path = No
Relative target port identifier = 6
Target port group identifier = 2

STATUS INFORMATION FOR LUN PATH : lunpath11

Generic Status Information

SCSI services internal state = UNOPEN
Open close state = DISABLE_OPENS
Protocol = fibre_channel
EVPD page 0x83 description code = 1
EVPD page 0x83 description association = 0
EVPD page 0x83 description type = 3
World Wide Identifier (WWID) = 0x6006016000e71e009c4e2eb7a794dd11
Total number of Outstanding I/Os = 0
Maximum I/O timeout in seconds = 30
Maximum I/O size allowed = 2097152
Maximum number of active I/Os allowed = 8
Maximum queue depth = 8
Queue full delay count = 0
Asymmetric state = NA
Device preferred path = No
Relative target port identifier = 1
Target port group identifier = 1

[Duncan Edmonstone · Accepted solution]

Scott,

Have you confirmed that the Storage team have set the right patameters for 11iv3 on the Clariion (host mode has to be ALUA). Also check they have upgraded the CLariion firmware to FLARE26 or higher. Post the output of:

scsimgr get_info -D /dev/rdisk/disk12

HTH

Duncan

[Scott Giddens]

Thank you, that is most likely something they are unaware of, I will ask them to verify that. Here is the output you offered to look at:

root@NASTRAN7> scsimgr get_info -D /dev/rdisk/disk12

STATUS INFORMATION FOR LUN : /dev/rdisk/disk12

Generic Status Information

SCSI services internal state = UNOPEN
Device type = Direct_Access
EVPD page 0x83 description code = 1
EVPD page 0x83 description association = 0
EVPD page 0x83 description type = 3
World Wide Identifier (WWID) = 0x6006016000e71e009c4e2eb7a794dd11
Serial number = "APM00074801378"
Vendor id = "DGC "
Product id = "CX3-80WDR5 "
Product revision = "HP03"
Other properties = ""
SPC protocol revision = 4
Open count (includes chr/blk/pass-thru/class) = 0
Raw open count (includes class/pass-thru) = 0
Pass-thru opens = 0
LUN path count = 4
Active LUN paths = 2
Standby LUN paths = 0
Failed LUN paths = 2
Maximum I/O size allowed = 2097152
Preferred I/O size = 2097152
Outstanding I/Os = 0
I/O load balance policy = cl_round_robin
Path fail threshold time period = 0
Transient time period = 120
Tracing buffer size = 1024
LUN Path used when policy is path_lockdown = NA
LUN access type = NA
Asymmetric logical unit access supported = No
Asymmetric states supported = NA
Preferred paths reported by device = No
Preferred LUN paths = 0

Driver esdisk Status Information :

Capacity in number of blocks = 4294967296
Block size in bytes = 512
Number of active IOs = 0
Special properties =
Maximum number of IO retries = 45
IO transfer timeout in secs = 30
FORMAT command timeout in secs = 86400
START UNIT command timeout in secs = 60
Timeout in secs before starting failing IO = 120
IO infinite retries = true
root@NASTRAN7>

[Scott Giddens]

The CX3-80 firmware is at version 26 but the storage guys don't know what ALUA is nor do they see where to set it. Attached is an image of all the settings they can set.

I believe the "HP no Trespass" should be set to "HP Trespass" is incorrect. is this true?

[kevin_m]

My company is using our CX-500 in failover mode 4. I don't know how to configure that (done by the SAN guru) but it's required for 11.31 multipathing with the CX. I attached an EMC doc for details (see page 42).

- Kevin

[kevin_m]

Attached a better doc that relates specifically to ALUA.

[Scott Giddens]

I added a prefered path and had the storage guys select "HP Trespass" for the host mode and I can create the VG, LV, and mount it. it shows all four paths active.

Thanks!

[Duncan Edmonstone]

Be carefull... there are modes that will "work" (in that you'll be able to create a VG/LV on them), but will deliver *terrible* performance. You need to make sure you have this set correctly.

I'm slightly surprised your Storage guys have never heard of ALUA as it is discussed in the EMC HP-UX connectivity guide in the Clariion Section. Here are some of the reelevant points:

-----------------------------------------
To run 11i v3 with CLARiiON arrays you must have the following
prerequisites:
â The CLARiiON array (CX3 series, CX300, CX500, and CX700)
must be running the Flare-Operating-Environment release 26 or
later and is required to support HP-UX 11iv3. This version of
Flare introduced the ALUA support for HP-UX 11i v3.
â ALUA support was introduced into HP-UX 11iv3 (11.31) with the
September 2007 11iv3 release. The host must be running the
March 2008 release of the 11iv3 operating system.
â Configure the initiators for ALUA failover mode (failover mode
4).
â CLARiiON Open and HP No Auto Trespass initiator types are
supported with HP-UX 11iv3 (11.31).
CLARiiON Open initiator type implements peripheral device
addressing resulting in expected HP-UX behavior of creating only
eight legacy device special files (DSFs) per target port.
HP No Auto Trespass initiator type implements the Volume Set
Addressing method. You must configure only HP No Auto
Trespass if there are any legacy DSFs dependencies that have the
format /dev/dsk/c2t1d0 and /dev/rdsk/c2t1d0.
You may configure either CLARiiON Open or HP No Auto
Trespass if there are no legacy DSFs dependencies and only
persistent DSFs (also known as agile DSFs) are utilized that have
the format /dev/disk/disk42 and /dev/rdisk/42.
-----------------------------------------
Using the Naviseccli command line
To use Naviseccli, you must have:
â Naviseccli installed on a host with network access to the
CLARiiON array
â Navisphere Agent running on the 11iv3 host you are connecting
to the CLARiiON array.
The following example uses the command line to configure the array:
naviseccli -h 172.23.180.130 -h 172.23.180.131 storagegroup -sethost -host
hpint064 -failovermode 4 â arraycommpath 1 -type 10
WARNING: Changing configuration options may cause the array to stop functioning
correctly. Do you wish to continue (y/n)? y
The syntax for the command is as follows:
-h 172.23.179.130 â the array ips
-host hpint064 â the host name (provided by Navisphere
Agent)
-failovermode 4 â the failover mode for ALUA support
-type 10 â the host initiator options
For ALUA, the only option (other than host) that can be modified for
support is the â typeâ options.
There are two supported values for the initiator options:
â 10
Using 10 sets the system for HP No Auto Trespass initiator
options.
â 3
Using 3 sets the system for CLARiiON Open initiator options.
-----------------------------------------


HTH

Duncan

[Duncan Edmonstone]

Scott,

Here's an output from an EVA LUN that also uses ALUA... I'd expect a Clariion to look similar:

# scsimgr get_info -D /dev/rdisk/disk19

STATUS INFORMATION FOR LUN : /dev/rdisk/disk19

Generic Status Information

SCSI services internal state = ONLINE
...
Vendor id = "HP "
Product id = "HSV300 "
Product revision = "0900"
...
LUN path count = 4
Active LUN paths = 2
Standby LUN paths = 2
...
LUN access type = T10 Asymmetric Active-Active
Asymmetric logical unit access supported = Both implicit and explicit
...


I've cut out the non-relevant sections, but note that this shows that half of the paths are in standb mode, and note the LUN access type and ALUA supported fields. I'd expect your array to look similar but with more paths in total?

HTH

Duncan

[Scott Giddens]

How did you configure two of the four paths for "standby"?

That appears to be the issue here:

NASTRAN7> scsimgr -f activate -D /dev/rdisk/disk12
scsimgr: The ioctl call for activate standby paths failed with return value of 22 (Invalid argument) for /dev/rdisk/disk12.
The error is returned for one of the following reasons:
1. LUN is neither asymmetric nor active-passive device
2. There are no standby paths available to the device
3. Device do not support manual activation of standby paths
4. Asymmetric access is disabled on the LUN or DDR


NASTRAN7> scsimgr lun_map -D /dev/rdisk/disk12

LUN PATH INFORMATION FOR LUN : /dev/rdisk/disk12

Total number of LUN paths = 4

LUN path : lunpath9
State = ACTIVE
Last Open or Close state = ACTIVE

LUN path : lunpath8
State = ACTIVE
Last Open or Close state = ACTIVE

LUN path : lunpath10
State = ACTIVE
Last Open or Close state = ACTIVE

LUN path : lunpath11
State = ACTIVE
Last Open or Close state = ACTIVE

[Duncan Edmonstone]

>> How did you configure two of the four paths for "standby"?

- I didn't - they were detected as that by dint of being on an ALUA disk array - if all 4 of yours are showing as "active" that suggests to me you may not have the host mode set correctly on the Clariion - I would expect some of them to show as "standby" (although I'm not completely sure as I don't have a Clariion to compaer with).

Have you confirmed you are using host mode 4 as outlined in some of the previous posts?

HTH

Duncan

[Duncan Edmonstone]

... sorry, not "host mode", but "failover mode"...

HTH

Duncan

[kevin_m]

Below is the lun_map output from a CX500 EMC disk using failover mode 4. We trespassed the LUN as a test and it failed over to the standby path temporarily then returned to the active path afterwards.


LUN PATH INFORMATION FOR LUN : /dev/rdisk/disk120

Total number of LUN paths = 4
World Wide Identifier(WWID) = 0x60060160f8c310006c73f45a4fc4dc11

LUN path : lunpath60
Class = lunpath
Instance = 60
Hardware path = 0/3/1/0.0x5006016930212882.0x400b000000000000
SCSI transport protocol = fibre_channel
State = ACTIVE
Last Open or Close state = ACTIVE

LUN path : lunpath170
Class = lunpath
Instance = 170
Hardware path = 0/3/1/0.0x5006016030212882.0x400b000000000000
SCSI transport protocol = fibre_channel
State = STANDBY
Last Open or Close state = STANDBY

LUN path : lunpath66
Class = lunpath
Instance = 66
Hardware path = 0/6/1/0.0x5006016830212882.0x400b000000000000
SCSI transport protocol = fibre_channel
State = ACTIVE
Last Open or Close state = ACTIVE

LUN path : lunpath182
Class = lunpath
Instance = 182
Hardware path = 0/6/1/0.0x5006016130212882.0x400b000000000000
SCSI transport protocol = fibre_channel
State = STANDBY
Last Open or Close state = STANDBY

- Kevin

[Scott Giddens]

failover mode 4? Sorry, I didn't understand the host mode reference and skipped over that, mostly because I am reading the "EMC Host Connectivity Guide for HP-UX" and there is no mention of failover mode 4.

So I assume it should be set to "HP Auto Trespass" and failover mode 4 at the same time?

[kevin_m]

I attached the ALUA doc again which explains how to change the host initiator failover mode. When using mode 4 the auto-trespass setting will be ignored.

[Duncan Edmonstone]

Scott - I guess you have an out of date copy of the host connectivity guide, as the lines I posted were from the same guide and talk about failover mode 4. My copy is REV A16 dated July 2008.

HTH

Duncan

[Scott Giddens]

Thank you!

my storage group gave me an old copy. I was able to download the ALUA white paper you last posted, prior to that my browser crashed when I selected to open it.

My storage group also sent me an image showing only up to mode 3, I suspect they are not at release 26 as they indicated to me they were.

Unless they need to do something additional to activate the ALUA asymmetric active/active feature (mode 4) it may be a while before they are able to update the firmware.

I asked them to contact EMC for specific help with this issue.

Thanks again,
Scott

[Scott Giddens]

Turns out the storage group was incorrect when they told me they were on FLARE 26. Not sure how they could have made that mistake but I think I can say the problem was figured out.

Thanks everyone!
Scott

[Scott Giddens]

Thread closed

[Duncan Edmonstone]

Scott,

If you're not on GLARE26 but wanted an "unsupported" method to use 11iv3, have a look at this post:

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1135213

HTH

Duncan

[Scott Giddens]

I saw that thread but it does not look like it is fully supported nor was there any results posted of the performance I need, and planned, for asymmetric load balancing of a common storage volume on an Infiniband fabric with a large number of Itanium 2 nodes. If it cannot handle the bandwidth requirements then my targeted cost/benefit of increasing the size of the cluster is lost. I can live with local SAS RAID 0 volumes until the storage group upgrades to flare 26 in 1 month.