Operating System - HP-UX
1752735 Members
5610 Online
108789 Solutions
New Discussion юеВ

HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

 
SOLVED
Go to solution
David Libby_4
Regular Advisor

HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

Hi,

We're just trying to make an HP-UX 11.31 installation talk correctly with our EMC Clariion CX700. We see the LUNS we presented but unfortunately the performance is horrific (400mb copied from vg00 to a vg on the SAN took 30 mins!!!!).

Does anybody out there have experience with HP-UX 11iv3 (11.31) and EMC Clariions?

Documentation is very sparce on both HP and EMC sites.

Hardware is rp3440...

Regards,
David Libby
20 REPLIES 20
Steven E. Protter
Exalted Contributor

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

Shalom David,

EMC has been known to ship those EMC boxes with incorrect parameters. I'd suggest taking a look at an EMC that performs properly and emulating it s configuration. That might find you an EMC configuration problem.

I'm aware of a business that spent $1,000,000 diganosing a software problem in its product when the actual cause was an EMC tech decided to make a bad configuration combination. I/O was awful.

SEP
Steven E Protter
Owner of ISN Corporation
http://isnamerica.com
http://hpuxconsulting.com
Sponsor: http://hpux.ws
Twitter: http://twitter.com/hpuxlinux
Founder http://newdatacloud.com
melvyn burnard
Honored Contributor

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

having seen some basics tests done using HP arrays and the native multipathing, I have to agree with SEP's suggestion, talk to your EMC specialist
My house is the bank's, my money the wife's, But my opinions belong to me, not HP!
David Libby_4
Regular Advisor

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

Hmmm.... well thanks for responding in any case.

Considering that that our Clariion has been in production for several years now and has a large number of machines connected to it (including a lot of HP-UX 11iv1 & 11iv2)... I would be disinclined to agree with your proposals.

We currently have support cases open with both vendors.

To EMC's credit... if HP are going to declare that third party "multi-pathing" products such as PowerPath are obsolete with 11iv3(which they have done), then they had better make sure their product works out-of-the-box (not done).

In any case discussions regarding the quality of support HP & EMC provide are best left aside.

If anyone out there actually has any *technical* answers or clues they'd be much appreciated.

Best regards,
David

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

David,

And where exactly did you read that the Clariion CX700 is actually supported with 11iv3?

Personally that would have been the first thing I wanted to check prior to using a brand new OS like 11iv3...

For your information, HP don't and never will offer support for Clariion disk arrays - thats EMC's problem - you'd get the same response from IBM, Sun or any other OS/Server vendor on that. What HP do carry out (or allow a third party to carry out) is 'independent interoperability validation', which means they work with EMC to 'qualify' their storage with HPUX and HP9000/Integrity. You can see the support matrix for that here:

http://www.hp.com/products1/serverconnectivity/mass_storage_devices.html#EMC

Note that 11iv3 is not mentioned.

Now EMC also carry out their own Interoperability tests and publish these on their customer support website (powerlink.emc.com) - I can't post a link to these here (you need a login), but I can tell you that this also doesn't mention support for 11iv3 with Clariion (it does for Symmetrix/DMX though).

So if neither the OS vendor nor the Array vendor list this as supported its not really a surprise that there are problems.

Now on to *why* there are problems:

The EMC Clariion is one of the few arrays on the market that still operates in an 'active passive' configuration - that is for any given LUN, IOs are only accepted on one controller (the EVA used to work like that as well about 3 years ago). What happens when an IO is sent to the non-owning controller (this event is called a trespass) depends on how a certain parameter is set on the Clariion (IIRC its called 'failover mode').

Now I expect that you have your failover mode set such that every time a trespass occurs, the owning controller for a LUN is changed - this operation is *slow* and if you are doing load balancing (which 11iv3 will do by default) almost every IO is going to cause a trespass.

Now you *might* be able to reolve this somewhat by playing with the load balancing options in the new storage stack (if you set load balancing to preferred path and then set the preferred path to the owning controller using the scsimgr command then at least *most* of the IOs would go to the owning controller, but anything that probes the other path is still going to cause a path failover and slow things down) - or I think theres an option to disable the new load balancing algorithms on the legacy DSFs (thats the cXtYdZ DSFs) - you could do that and then just use LVM PVLinks which were ceratinly supported in 11iv1/11iv2 - but that doesn't alter the fact that you are running an unsupported configuration...

I expect EMC are working on a plug-in or a version of PowerPath for 11iv3 which will resolve this, so maybe you can persuade them to give you a beta, otherwise I guess you need to go back to 11iv2 or use another storage array.

HTH

Duncan

I am an HPE Employee
Accept or Kudo
Geoff Wild
Honored Contributor

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

I'm currently benchmarking a RX6600.

30mins - yes - there is definately something very wrong there...

Now mine isn't on a clariion - it's on a DMX1000 and DMX3

From the output below, frame 65 is the 1000 and 52 is the DMX3.

# prealloc /tmp/bigfile 409600000

# timex cp /tmp/bigfile /v01/

real 11.95
user 0.00
sys 0.18

How did you create your vg?

Can you post the commands I ran below for your env?

I did mine like so:

mkdir /dev/vg11
mknod /dev/vg11/group c 64 0x0b0000

vgcreate -s 32 -p 64 -e 32768 /dev/vg11 /dev/disk/disk23 /dev/disk/disk24 /dev/disk/disk25



# ioscan -m dsf
Persistent DSF Legacy DSF(s)
========================================
/dev/pt/pt1 /dev/rscsi/c1t0d0
/dev/pt/pt3 /dev/rscsi/c3t0d0
/dev/rdisk/disk4 /dev/rdsk/c0t1d0
/dev/rdisk/disk4_p3 /dev/rdsk/c0t1d0s3
/dev/rdisk/disk4_p1 /dev/rdsk/c0t1d0s1
/dev/rdisk/disk4_p2 /dev/rdsk/c0t1d0s2
/dev/pt/pt5 /dev/rscsi/c5t0d0
/dev/rdisk/disk5 /dev/rdsk/c0t0d0
/dev/rdisk/disk5_p1 /dev/rdsk/c0t0d0s1
/dev/rdisk/disk5_p2 /dev/rdsk/c0t0d0s2
/dev/rdisk/disk5_p3 /dev/rdsk/c0t0d0s3
/dev/pt/pt7 /dev/rscsi/c7t0d0
/dev/rdisk/disk7 /dev/rdsk/c2t0d0
/dev/rdisk/disk9 /dev/rdsk/c4t0d0
/dev/rdisk/disk18 /dev/rdsk/c4t9d0
/dev/rdisk/disk19 /dev/rdsk/c4t9d1
/dev/rdisk/disk20 /dev/rdsk/c4t9d2
/dev/rdisk/disk21 /dev/rdsk/c4t9d3
/dev/rdisk/disk22 /dev/rdsk/c2t9d0
/dev/rdisk/disk23 /dev/rdsk/c2t9d1
/dev/rdisk/disk24 /dev/rdsk/c2t9d2
/dev/rdisk/disk25 /dev/rdsk/c2t9d3
/dev/rdisk/disk27 /dev/rdsk/c6t0d0
/dev/rdisk/disk29 /dev/rdsk/c8t0d0
/dev/rdisk/disk38 /dev/rdsk/c8t4d0
/dev/rdisk/disk39 /dev/rdsk/c8t4d2
/dev/rdisk/disk40 /dev/rdsk/c8t3d7
/dev/rdisk/disk41 /dev/rdsk/c8t4d1
/dev/rdisk/disk42 /dev/rdsk/c6t4d0
/dev/rdisk/disk43 /dev/rdsk/c6t3d7
/dev/rdisk/disk44 /dev/rdsk/c6t4d1
/dev/rdisk/disk45 /dev/rdsk/c6t4d2



# inq
Inquiry utility, Version V7.3-771 (Rev 0.0) (SIL Version V6.3.0.0 (Edit Level 771)
Copyright (C) by EMC Corporation, all rights reserved.
For help type inq -h.

............................

-----------------------------------------------------------------------------
DEVICE :VEND :PROD :REV :SER NUM :CAP(kb)
-----------------------------------------------------------------------------
/dev/rdsk/c0t0d0 :HP :DG072ABAB3 :HPD5 :3NP0D3VQ : 71687369
/dev/rdsk/c0t0d0s1 :HP :DG072ABAB3 :HPD5 :3NP0D3VQ : 71687369
/dev/rdsk/c0t0d0s2 :HP :DG072ABAB3 :HPD5 :3NP0D3VQ : 71687369
/dev/rdsk/c0t0d0s3 :HP :DG072ABAB3 :HPD5 :3NP0D3VQ : 71687369
/dev/rdsk/c0t1d0 :HP :DG072ABAB3 :HPD5 :3NP0BTRC : 71687369
/dev/rdsk/c0t1d0s1 :HP :DG072ABAB3 :HPD5 :3NP0BTRC : 71687369
/dev/rdsk/c0t1d0s2 :HP :DG072ABAB3 :HPD5 :3NP0BTRC : 71687369
/dev/rdsk/c0t1d0s3 :HP :DG072ABAB3 :HPD5 :3NP0BTRC : 71687369
/dev/rdsk/c2t9d0 :EMC :SYMMETRIX :5670 :6500780000 : 35354880
/dev/rdsk/c2t9d1 :EMC :SYMMETRIX :5670 :650079c000 : 35354880
/dev/rdsk/c2t9d2 :EMC :SYMMETRIX :5670 :6500774000 : 35354880
/dev/rdsk/c2t9d3 :EMC :SYMMETRIX :5670 :650077c000 : 35354880
/dev/rdsk/c4t9d0 :EMC :SYMMETRIX :5670 :6500780000 : 35354880
/dev/rdsk/c4t9d1 :EMC :SYMMETRIX :5670 :650079c000 : 35354880
/dev/rdsk/c4t9d2 :EMC :SYMMETRIX :5670 :6500774000 : 35354880
/dev/rdsk/c4t9d3 :EMC :SYMMETRIX :5670 :650077c000 : 35354880
/dev/rdsk/c6t0d0 :EMC :SYMMETRIX :5771 :5200022000 : 5760
/dev/rdsk/c6t3d7 :EMC :SYMMETRIX :5771 :52000e3000 : 77760000
/dev/rdsk/c6t4d0 :EMC :SYMMETRIX :5771 :52000e7000 : 77760000
/dev/rdsk/c6t4d1 :EMC :SYMMETRIX :5771 :52000eb000 : 77760000
/dev/rdsk/c6t4d2 :EMC :SYMMETRIX :5771 :52000ef000 : 77760000
/dev/rdsk/c8t0d0 :EMC :SYMMETRIX :5771 :5200022000 : 5760
/dev/rdsk/c8t3d7 :EMC :SYMMETRIX :5771 :52000e3000 : 77760000
/dev/rdsk/c8t4d0 :EMC :SYMMETRIX :5771 :52000e7000 : 77760000
/dev/rdsk/c8t4d1 :EMC :SYMMETRIX :5771 :52000eb000 : 77760000
/dev/rdsk/c8t4d2 :EMC :SYMMETRIX :5771 :52000ef000 : 77760000


# vgdisplay -v vg11
--- Volume groups ---
VG Name /dev/vg11
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 1
Open LV 1
Max PV 64
Cur PV 3
Act PV 3
Max PE per PV 32768
VGDA 6
PE Size (Mbytes) 32
Total PE 3234
Alloc PE 3234
Free PE 0
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vg11/lvdata01
LV Status available/syncd
LV Size (Mbytes) 103488
Current LE 3234
Allocated PE 3234
Used PV 3


--- Physical volumes ---
PV Name /dev/disk/disk23
PV Status available
Total PE 1078
Free PE 0
Autoswitch On

PV Name /dev/disk/disk24
PV Status available
Total PE 1078
Free PE 0
Autoswitch On

PV Name /dev/disk/disk25
PV Status available
Total PE 1078
Free PE 0
Autoswitch On


Rgds...Geoff
Proverbs 3:5,6 Trust in the Lord with all your heart and lean not on your own understanding; in all your ways acknowledge him, and he will make all your paths straight.
David Libby_4
Regular Advisor

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

Hi Geoff,

We've got 3 VGs configured on that machine... but we only need one for now as they were all setup using the same commands.

The usual mkdir/mknod and vgcreate scenario... nothing special.

Here we go... for vge1
======================================
root@s000311 # ioscan -m dsf
Persistent DSF Legacy DSF(s)
========================================
/dev/pt/pt2 /dev/rscsi/c4t0d0
/dev/rscsi/c6t0d0
/dev/rdisk/disk13 /dev/rdsk/c5t0d0
/dev/rdsk/c7t0d0
/dev/rdisk/disk14 /dev/rdsk/c5t0d1
/dev/rdsk/c7t0d1
/dev/rdisk/disk15 /dev/rdsk/c5t0d2
/dev/rdsk/c7t0d2
/dev/rdisk/disk16 /dev/rdsk/c5t0d3
/dev/rdsk/c7t0d3
/dev/rdisk/disk17 /dev/rdsk/c3t3d0
/dev/rdisk/disk18 /dev/rdsk/c3t2d0
/dev/rdisk/disk19 /dev/rdsk/c3t1d0
/dev/rdisk/disk20 /dev/rdsk/c2t0d0
/dev/rdisk/disk21 /dev/rdsk/c0t0d0
/dev/rdisk/disk24 /dev/rdsk/c5t0d7
/dev/rdsk/c7t0d7
/dev/rdisk/disk26 /dev/rdsk/c2t1d0
root@s000311 # inq
Inquiry utility, Version V7.3-782 (Rev 1.0) (SIL Version V6.3.1.0 (Edit Level 782)
Copyright (C) by EMC Corporation, all rights reserved.
For help type inq -h.

................

---------------------------------------------------------------------------
DEVICE :VEND :PROD :REV :SER NUM :CAP(kb)
---------------------------------------------------------------------------
/dev/rdsk/c0t0d0 :TEAC :DV-28E-N :E.6A : : -----
/dev/rdsk/c2t0d0 :HP 146 G:MAT3147NC :HPC2 :AX009940 : 143374744
/dev/rdsk/c2t1d0 :HP 146 G:MAT3147NC :HPC2 :AX008896 : 143374744
/dev/rdsk/c3t1d0 :HP :DVD-ROM 305 : : : -----
/dev/rdsk/c3t2d0 :HP :DVD-ROM 305 : : : -----
/dev/rdsk/c3t3d0 :HP :DVD-ROM 305 : : : -----
/dev/rdsk/c5t0d0 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 35354880
/dev/rdsk/c5t0d1 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 125829120
/dev/rdsk/c5t0d2 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 157286400
/dev/rdsk/c5t0d3 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 157286400
/dev/rdsk/c5t0d7 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 10485760
/dev/rdsk/c7t0d0 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 35354880
/dev/rdsk/c7t0d1 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 125829120
/dev/rdsk/c7t0d2 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 157286400
/dev/rdsk/c7t0d3 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 157286400
/dev/rdsk/c7t0d7 :DGC :CX700WDR5 :HP03 :Ch2 CONT : 10485760
root@s000311 # vgdisplay -v vge1
--- Volume groups ---
VG Name /dev/vge1
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 2
Open LV 2
Max PV 64
Cur PV 1
Act PV 1
Max PE per PV 20000
VGDA 2
PE Size (Mbytes) 16
Total PE 2156
Alloc PE 1876
Free PE 280
Total PVG 0
Total Spare PVs 0
Total Spare PVs in use 0

--- Logical volumes ---
LV Name /dev/vge1/lv101
LV Status available/syncd
LV Size (Mbytes) 5008
Current LE 313
Allocated PE 313
Used PV 1

LV Name /dev/vge1/lv102
LV Status available/syncd
LV Size (Mbytes) 25008
Current LE 1563
Allocated PE 1563
Used PV 1


--- Physical volumes ---
PV Name /dev/disk/disk13
PV Status available
Total PE 2156
Free PE 280
Autoswitch On


root@s000311 #
=========================================

Regards,
David
Geoff Wild
Honored Contributor
Solution

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

I just spoke with our SAN Admin - he's allocating ne a LUN from our Clariion for testing - one thing he mentioned - make sure you have configured the initiator on the Clariion correctly - IE for DMP.

Rgds...Geoff
Proverbs 3:5,6 Trust in the Lord with all your heart and lean not on your own understanding; in all your ways acknowledge him, and he will make all your paths straight.
David Libby_4
Regular Advisor

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

Hi Geoff,

We're just checking the Clariion initiator setting... for DMP if I have understood correctly it seems that it should be "No Auto-Trespass" + "Failover Mode"=2. We're just trying that now... thanks a lot.

Cheers,
David
David Libby_4
Regular Advisor

Re: HP-UX 11.31 native multi-pathing and EMC Clariion CX700...

Duncan,

Thanks for your amazingly detailed and accurate response! It's equivalent to flapping in a APA network aggregate if I have understood correctly? In any case, regardless of who supports what it looks like we're on our way to finding the right combination of settings... will give feedback tomorrow mornign after more tests and will of course post the exact "unsupported" solution.

Best regards,
David