HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

[Santosh Rao]

Duncan has given a very accurate response.

11iv3 native multipathing supports active-active arrays transparently without needing any vendor supplied plug-ins.

Active-passive arrays need to either support ALUA (See Section 5.8 of http://t10.org/ftp/t10/drafts/spc3/spc3r23.pdf) or the active-passive array vendor needs to supply an apsw plug-in for their array. See :
http://h21007.www2.hp.com/portal/download/files/unprot/ddk/APSWWPAPER.pdf

You may want to enquire with your EMC sales rep on when EMC will support either ALUA or will provide the apsw active-passive plug-in for support of HP-UX 11iv3.

[David Libby_4]

Okay everone, here goes with our home-brewed, unsupported configuration... which works for now.

Config for Clariion-side CX700:
-------------------------------
- We must first configure the two 2 initiators for our server in the following way on the Clariion:

============
Initiator Type: HP No Auto Trespass
Failover Mode: 2
============

Config for Server-side (HP-UX 11.31):
--------------------------------------
In our current configuration, we have 2 HBAs: the first is hooked up to SP-A on the Clariion and the second, SP-B. As already established, it's not possible to perform load-balancing, because the Clariion CX700 fonctions in "Active/Passive" mode. Therefore we have to change the load-balancing policy for each LUN in order to replicate a basic failover mode (hot spare if you like).

To do that we need to use the "preferred_path" argument for scsimgr.

First though we need the hardware adress in hex... ugh!

To get that we run ioscan with the following args.
==================================================================
# ioscan -fnNkCdisk
Class I H/W Path Driver S/W State H/W Type Description
===================================================================
disk 13 64000/0xfa00/0x1 esdisk CLAIMED DEVICE DGC CX700WDR5
/dev/disk/disk13 /dev/rdisk/disk13
disk 14 64000/0xfa00/0x2 esdisk CLAIMED DEVICE DGC CX700WDR5
/dev/disk/disk14 /dev/rdisk/disk14
disk 15 64000/0xfa00/0x3 esdisk CLAIMED DEVICE DGC CX700WDR5
/dev/disk/disk15 /dev/rdisk/disk15
disk 16 64000/0xfa00/0x4 esdisk CLAIMED DEVICE DGC CX700WDR5
/dev/disk/disk16 /dev/rdisk/disk16
disk 20 64000/0xfa00/0x8 esdisk CLAIMED DEVICE HP 146 GMAT3147NC
/dev/disk/disk20 /dev/rdisk/disk20
disk 21 64000/0xfa00/0x9 esdisk CLAIMED DEVICE TEAC DV-28E-N
/dev/disk/disk21 /dev/rdisk/disk21
disk 26 64000/0xfa00/0xc esdisk CLAIMED DEVICE HP 146 GMAT3147NC
/dev/disk/disk26 /dev/rdisk/disk26
disk 24 64000/0xfa00/0xd esdisk CLAIMED DEVICE DGC CX700WDR5
/dev/disk/disk24 /dev/rdisk/disk24
==================================

Then we fix the so called "load-balancing" policy...
==================================
/usr/sbin/scsimgr save_attr -H 64000/0xfa00/0xd -a load_bal_policy="preferred_path"
==================================

To find out the exact hardware path to "fix" as "preferred" you need to use the following command....
==================================
# /usr/sbin/scsimgr lun_map -D /dev/rdisk/disk24

LUN PATH INFORMATION FOR LUN : /dev/rdisk/disk24

Total number of LUN paths = 2
World Wide Identifier(WWID) = 0x60060160ce0b1600a242ce17f917dc11

LUN path : lunpath16
Class = lunpath
Instance = 16
Hardware path = 0/3/1/0/4/0.0x5006016930600f94.0x4007000000000000
SCSI transport protocol = fibre_channel
State = ACTIVE
Last Open or Close state = ACTIVE

LUN path : lunpath15
Class = lunpath
Instance = 15
Hardware path = 0/4/1/0/4/0.0x5006016130600f94.0x4007000000000000
SCSI transport protocol = fibre_channel
State = ACTIVE
Last Open or Close state = ACTIVE
=================================

After activating that policy we now force the path for the LUN to be point to the corresponding SP on the Clariion. Like so:
=================================
/usr/sbin/scsimgr save_attr -D /dev/rdisk/disk24 -a preferred_path=0/3/1/0/4/0.0x5006016930600f94.0x4007000000000000
=================================

We have now tested this with some large file copies and it performs just as well as on other machines connected with a single HBA (was not the case before).

We have also tested the failover of HBAs by forcing a trespass on the Clariion (in both directions)... no adverse effects were perceived.

[Geoff Wild]

David,

I have this update from our EMC rep:

"HPUX 11.31 will only work with active/active arrays. CLARiiON arrays are active/passive. We already support native MPIO for Sun and Windows. For HP/UX, we can't promise anything since the ball is mostly in HP's court. We hope so but looks like it will be a patch to 11iv3 to enable it.......along with release 26 on the CLARiiON.

Lastly, let me say this. I've been told if you move ahead (ie above) on anything other than R26, you will have data corruption issues. "

Rgds...Geoff

[David Libby_4]

Thanks Geoff... I'm going to wait for official confirmation from our EMC reps over here and the closure of the support call we have with them regarding it. If that statement is corroborated over here, I suspect there's going to be some serious heat travelling towards EMC/HP from our mgmt. In any case it's all a very interesting excercise... if the risk is confirmed then we will stick with 11.23 or attach the concerned machines to a Symmetrix.

[Santosh Rao]

Hi,

EMC rep is only partially correct.

HP-UX 11.31 does support active/passive as well as asymmetric active-active arrays. However, it requires the array vendor to support ALUA or to provide the apsw plug-in. ALUA is already supported by a number of the well known mid-range asymmetric active-active arrays. The reference to upcoming rev 26 should be enabling ALUA and with that, the Clariion will work transparently with HP-UX 11.31.

You should see transparent support for the Clariion once ALUA is out in a couple of months. In the meantime, the use of preferred path as previously described on this thread is a reasonable solution as long as you are using a single host configuration (i.e. not in a SG cluster, not having multiple host OS' connected to the same luns). To use the interim solution with preferred path, you will need to configure the Clariion in Auto Tresspass mode.

Another option is to use the legacy device files only on 11.31, set the leg_mpath_enable attribute to 0 and configure/use pvlinks.

scsimgr get_attr -a leg_mpath_enable
scsimgr set_attr -a leg_mpath_enable=0
scsimgr get_attr -a leg_mpath_enable

Thanks,
Santosh

[Otero Tony]

Hi David,

I'm in the same situation with a CX500. I have done what you mentionned (preferred path) and now it seems that the LUN is not trespassing aymore. So, the performances will probably improve.
I will probably keep the machine with that HP-UX version. That machine will be in production in September (1st) and I'm a little affraid about the possible issues I could get. Have you any information about any patch, from HP or EMC to solve this? What are you going to do: are you also going to use HP-UX 11.31 or are you going to the V2 version? I would like to keep that version, but I don't know if that is too risky...

Thanks a lot for openning that topic, because it helped me a lot.

[Alzhy]

Greetings!

I am in the same boat. HDS9500V series arrays. No official support from HP nor HDS (although internal testing probably shows it is supported but..)

I followed the Libby Recipe and our dual Fibre Connected 9500Vs now work like a charm.

I use VxVM 5.0 on 11.31 which "should" support the 9500V series as an ALUA array but it does not seem to behave properly (very poor performance until the path preferencing trick above).

# vxddladm listsupport libname=libvxhdsalua.sl
ATTR_NAME ATTR_VALUE
=======================================================================
LIBNAME libvxhdsalua.sl
VID HITACHI
PID DF600, DF600-V, DF600F
DF600F-V
ARRAY_TYPE A/A-A-HDS, A/PG, A/P
A/A
ARRAY_NAME HDS9500-ALUA, AMS_WMS

Maybe "FULL" support is coming soon?

[Alzhy]

BTW, the ASL library (aka Array Support Library) came bundled with VxVM 5.0.

On 11.11 - (VxvM 3.5) it requires a separate ASL .

Same on 11.23 (VxVM 4.1 and 5.0) -- it requires an ASL too.

What prompted me to think and to search there must be something to tweak MPIO wise was the fact that even under LVM and RAW device usage - performance was horrible -- massive queing and all...

Now we're a go go Good.

The only pain now is to go through each of the hundreds or so Hitachi 9500V LUN on my 11.31 Server (which happens to be our lone 11.31 server -- acting as the NetBackup Master). We use these 9500Vs as mirror pairs for our Tier-1 arrays that are hokked up to 11.11 HP-UX servers.


Danke ITRC once more...!

[Duncan Edmonstone]

Nelson,

>> The only pain now is to go through each of the hundreds or so Hitachi 9500V LUN on my 11.31 Server

Ahh now you'll get a chance to see the real "beauty" of the new IO stack to make at least part of this simpler for you

What you need to do is setup a new attribute scope with scsimgr which identifies only LUNs on the HDS9500V:

Identify a suitable DDR name using scsimgr ddr_name against one of the HDS LUNs (any will do) You'll have to decide whether you do this down to just the actual array level, or down to the array and firmware level (I'd suggest just to the array level myself), but anyway compare what you get from

scsimgr ddr_name -D /dev/rdisk/diskNN vid

and

scsimgr ddr_name -D /dev/rdisk/diskNN rev

You should get something like "/escsi/esdisk/0x0/HITACHI /DF600-V /0123"

or something like that (I don't have a HDS box to test against)

Then use one of those to define the scope:

scsimgr ddr_add -N "/escsi/esdisk/0x0/HITACHI /DF600-V /0123"

Now you have a scope which will effect all objects of that type (including new ones when they're added), and you can set attributes across that scope as in:

scsimgr save_attr -N "/escsi/esdisk/0x0/HITACHI /DF600-V /0123" -a load_bal_policy="preferred_path"

However as for setting the preferred path - well do you have an easy way of detrmining programatically which lunpath should be the preferred path (is it something really simple on the HDS array like each odd lun is on one controller an each even lun is on the other???) But all I can think here is that you're going to have to script it in some way (surely doing it by hand isn't an option??)


HTH

Duncan

[Alzhy]

Duncan,

Thanks for your input man but our 11.31 server hosts 4 9500V arrays. And I did ddr_name vid and rev on a disk from each HDS array but there does not seem to be a distinguishing differentiator to identify which is HDS1 2 3 or 4.

I am still awaiting word whetehr Hitachi has something similar to "xpinfo" or "evainfo" as that would be a great help.

BTW, the 9500V seems to be recognaized as an ALUA under VxVM 5.0 (althugh Symantec does not mention official support but their curent VxVM 5.0 seem to recognize it as an ALUA array and they have libraries, not addon ASL, built in).

# vxddladm listsupport libname=libvxhdsalua.sl
ATTR_NAME ATTR_VALUE
=======================================================================
LIBNAME libvxhdsalua.sl
VID HITACHI
PID DF600, DF600-V, DF600F
DF600F-V
ARRAY_TYPE A/A-A-HDS, A/PG, A/P
A/A
ARRAY_NAME HDS9500-ALUA, AMS_WMS