HPE EVA Storage

EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

 
McKDEVStorage
Advisor

EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

Hello All,

I've been working a project testing Redhat Multipath (device mapper) using all the storage arrays our company uses and supports. The two main arrays are the EMC Clariion CX3/4 products and the HP EVA8400. These are what I use to qualify products with.

I just completed the Clariion part and when you install it in a basic way only storage controller SPA paths 1 and 2 work at a given time. You'd have to disable both the SAN switch ports on SPA to make multipath move it over to controller SPB.

Then I read an article on EMC knowledgebase and added some recommended lines into the multipath.conf file as a DEVICE. Then removed and readded the MPATH1,2,3,4 devices and suddenly they now use ALL FOUR storage controller ports on SPA and SPB and performance is increased as well as load balancing functionality.

NOW fast forward to EVA8400 test. Completed the installation fine both manually and using the HP provided implementation script or kit. They do the SAME thing. In fact redhat device mapper's default multipath.conf.example file contains the same parameters as the ones the free HP script install. Which look like this and are indeed being used properly since the multipath -ll command confirms these settings do take effect for the EVA disks.

The problem is these settings mention ALUA and I swear I remember Secure Path with older EVA's pumping ALL paths to both HSV controllers. In fact I wrote an article on it when I worked for DEC/Compaq/HP.

The question for the day is HOW DO YOU GET PATHS working from both controllers servicing a lun to the SAN?? Perhaps the EVA is not TRUE active/active in the sense of a super high end array like the Hitachi/XP. However I thought the EVA supported both storage controllers or DAC's pumping data to and from a host to a LUN.

Has anyone here actually tested the paths after you installed one of these solutions. I know it's not very common for support or install groups to verify all paths work when they install but the devil is in the details.

In my organization HP is struggling to make headway with their storage line since we are a mega sized customer and partner. Having said that running the EVA on one controller only is not going to help sell this array on folks unless I can get this working.

Here's the params in multipath.conf that just don't seem to cut it. This is from the redhat example and also what the HP device mapper enablement kit script (latest 4.3.0) installed.

device {
vendor "(COMPAQ|HP)"
product "HSV1[01]1|HSV2[01]0|HSV300|HSV4[05]0"
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_alua /dev/%n"
features "0"
hardware_handler "0"
path_grouping_policy group_by_prio
failback immediate
rr_weight uniform
no_path_retry 12
rr_min_io 100
path_checker tur
}

THIS IS WHAT TAKES EFFECT

mpath5 (3600508b4000af0bb0000300000f60000) dm-3 HP,HSV450
[size=10G][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=100][active]
\_ 5:0:2:1 sdj 8:144 [active][ready]
\_ 6:0:0:1 sdv 65:80 [active][ready]
\_ round-robin 0 [prio=20][enabled]
\_ 6:0:1:1 sdab 65:176 [active][ready]
\_ 5:0:3:1 sdp 8:240 [active][ready]


Thanks for your valuable time,
D.
13 REPLIES 13
Michael Leu
Honored Contributor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

I'm no expert in this... this looks good to me.

EVA vDisks always have one managing/preferred controller. You can see this in Command View in the presentation tab of the vDisk. This becomes the preferred path with ALUA. IO to this vDisk via the other controller is slower. As you usually have more than one vDisk, roughly half are owned by one controller and the other half by the other controller.

With newer firmware the EVA automatically detects if a lot of IO is made over the slower controller and changes the ownership of that vDisk.
McKDEVStorage
Advisor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

Thanks for your prompt reply. Yes I can load balance LUNs across the A and B controllers and that's fine but I'm needing to see if true ALUA access is possible with the EVA.

Hopefully someone out there has been able to make both HSVs in an EVA pair service I/Os to a single LUN.

Thanks again for your time,
D.

Michael Leu
Honored Contributor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

You could just remove prio_callout and change path_grouping_policy to multibus in your multipath.conf...

Performance would probably suffer and the EVA would switch vDisk ownership like crazy ;-)
McKDEVStorage
Advisor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

We'll one thing I'm not wanting to do is test some unsupported config. What I'm hoping is that someone will have some valuable tips that are not documented. Also someone who has tested device mapper and knows the absolute optimal way to configure it.

For example here is an example of what I added to the EMC array that made ALUA start working. This is documented in an EMC knowledgebase article. If I can find the same kind of information for the EVA I would be happy. Or just confirming there is no possibility of running it with ALUA working.

EMC multipath.conf settings that make ALUA work and actually improves performance... even though talking through the backplane for some IOs. No LUN thrashing either.

#}
devices {
device {
vendor "DGC"
product "*"
path_grouping_policy group_by_prio
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
path_checker emc_clariion
path_selector "round-robin 0"
prio alua
hardware_handler "1 alua"
features "1 queue_if_no_path"
failback immediate
rr_weight priorities
no_path_retry 60
}
}
# device {


I'm going to seek some help through our partner channel to eng but if anyone has some direct experience they can share...

Thanks
skt_skt
Honored Contributor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

Even on the clariion if you see the four paths ONLy two are served at a time(either by SPA or SPB). SInce you have 2 HBA both connects to SPA&SPB you have 2 active paths and 2 passive paths with load balancing across the 2 HBA. This is how the Clar behaves even w/o ALUA. ALUA is supported in some env , not sure with RHEL 5. Which version of release 5?
McKDEVStorage
Advisor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

Well that's really not true. You are just wrong about this. I'm not guessing on this stuff though. I have a lab setup and I'm sniffing and recording all traffic. With the CORRECT settings in multipath.conf and making sure the EMC disks are properly detected from a reboot you will indeed see ONE single Clariion DGC RAID 5 lun talking to the host from all FOUR paths, two on SPA and two on SPB. If you dont have this feature enabled properly you will see all luns served out of SPA's two ports, or SPB's two ports with no exception. You can also make all the luns failover to the other controller at one time but they all stay lumped together.

My guess is there are a whole lot of Clariion's out there running at half their features because people are not reading the EMC article and updating these params. Sounds like you're a potential example of that....

McKDEVStorage
Advisor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

Sorry if I sound a little frustrated but I'm getting some runaround and conflicting answers from HP support, engineering, sales and I will likely have to sort this out with engineering directly. I just hoped I'd run into someone with direct knowledge and expertise that is current. I do appreciate getting a response but sounds like I'll need to dig this answer from another support channel. I need to nail this down with some certainty. Thanks again for trying to help.
skt_skt
Honored Contributor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

what is the fail over mode used in this test env(is it 1 or something else)?
McKDEVStorage
Advisor

Re: EVA8400 Redhat 5 getting all paths to work, ALUA & multipath.conf

Per EMC, the Clariion must have flare 29 and it's connections set to "failover mode 4" to go into ALUA mode. In fact EMC has a document in powerlink on failover modes and MPIO that contains this info but just having that turned on wont make ALUA work.

The key to making both controllers load balance IO to a single lun is enabling parameters in the device mapper's config file.

That's where an additional EMC article comes into play. It gives the RIGHT parameters for multipath.conf that enable ALUA. Once those are in place you have to /sbin/dmsetup remove mpath# and then reboot so the devices are rescanned and then when you multipath -ll you will see the parameters the device uses will have changed and the word "ALUA" is displayed as one of the parameters.

Then all ports on the Clariion light up and the load numbers are evenly divided. So if each port on SPA was running 100MBs before the change, then now, both controllers are running and 4 ports read 50MBs. So the overall benchmark for the LUN actually remains the same BUT the load is much better balanced as far as paths are concerned.