EVA, VMS Blade and preferred paths

[Robert Atkinson]

Hi Steve.

We've recently moved to async-CA between Essex and Lincs. Up until then, we'd come to the conclusion of NOT using SET DEV/PATH, as they get pushed around by VMS anyway.

However, because DR disks in a particular group all have to be delivered from the same controller, we've decided to add SET DEV/PATH from VMS for those disks (don't ask me why - I can't remember!).

Another piece of information you may not be aware of is that although the path may be switch from controller A to B, controller B will not necessarily pick up the load. Instead, controller B passes all requests to controller A, and it continues to do the work. If your EVA is heavily loaded, this can start to show.

Cheers, Rob.

[Uwe Zessin]

> active/active support, it's less of an issue, but I believe that there is still a distinction for a dual-controller setup, with a performance penalty on reads.

Right, because on mid-range arrays into which group the EVA belongs (despite the E-nterprise in its name) a single controller is responsible for all physical disk I/O of a particular virtual disk.

Does that mean that VMS still does not understand ALUA (asymmetric logical unit access)?
If it did, you should set up preferred paths (in reality it means you assign management of a virtual disk to a controller...) on the EVA and all VMS hosts should detect this and limit I/O to one of the 'performance paths'.

[JBF]

Steve,

Since V8.3 OpenVMS understands and uses Asymmetric Logical Unit Access (ALUA) with the EVA storage arrays.

OpenVMS will always choose a path based on which controller manages the vdisk. It also uses this information during a mount verification event. For example, if a path fails (such as when a fibre cable is removed) then OpenVMS will (1) first retry the current path, then (b) search for another Active Optimized (AO) path (on the same controller) to the LUN, and then if necessary choose an Active Not Optimized (ANO) path on any other controller that can access the LUN. If OpenVMS must move to an ANO path, then OpenVMS requests the new controller assume control of the LUN and make it an Active Optimized path.

Due to the coordination between nodes in a VMScluster, I do NOT reocmmend using preferred pathing on the EVA storage array. I have worked several cases where this led to problems. Instead, I recommend allowing OpenVMS to pick a path.

However, on top of that, I suggest you build a DCL command procedure to periodically perform a SET DEVICE/SWICH/PATH=. Ideally you will take the top "n" devices (in terms of the I/O request rate) and spread those across the available paths. This only needs to be done on one node in the VMScluster if you run OpenVMS V8.3 or later. And you probably only need to do this once a day.

Why do this? It helps avoid problems where all the LUNs are switched to one controller (such as when a switch or controller is rebooted). This will at least daily be certain your top workload is balanced across the paths.

Hope that helps!
=jbf=

John B. Fisher

[Robert Atkinson]

Interesting reading John. I'd heard that VMS had become more 'aware' of the controllers, but this is the first information I've seen to back that up.

Could you expand a little more on "Due to the coordination between nodes in a VMScluster, I do NOT reocmmend using preferred pathing on the EVA storage array"

Why is this important, and how would it effect the use of DR disk groups?

Thanks, Rob.

[Ian Miller.]

will multipath polling switch to a AO path from a ANO path or does the polling just check that the path works?

[Robert Brooks_1]

will multipath polling switch to a AO path from a ANO path or does the polling just check that the path works?

--

The multipath poller is very lightweight. The entity that's relevant is the path, not any one device. If *any* device is reachable on a path, that path is deemed "working".

The details for those with the sources:

Take a look at the mpdev_ppb structure in [lib]
and the code in [sys]mpdev_poller.c. You'll see that the mpdev_ppb structure maintains a list of all devices on a path; thus, there is an mpdev_ppb structure for each path on a system.

The poller walks this list of devices, sending an io$_path_verify I/O. If the first device responds correctly, the poller for that path terminates, and waits for the next interval (usually 60 seconds, if a path is OK). If the first device has "issues", then we'll try the next device. Only after *every* device on a path has failed do we declare the path "bad", and we emit an OPCOM message to that effect.

The poller does not affect current paths for any device *except* for the case of "failback" from an MSCP path. If we've failed over to an MSCP path because all fibre paths are down, it's the poller's job to notice that a device that automatically switched to the MSCP path can now be "failed back" to a fibre path. In that case, the poller will initiate a path switch away from the MSCP path. If, for some reason, the MSCP path was manually selected, the poller will leave it on the MSCP path.

-- Rob

[Ian Miller.]

There does not appear to be a way of telling VMS to re-evaluate the paths and choose the 'best'

[EdgarZamora_1]

In reading this thread, John (Fisher) mentions that OpenVMS 8.3 is ALUA-aware. I was wondering where I could find documentation on this. Unless I missed it, I didn't find it anywhere in the OpenVMS docs.

I guess I will do some actual experimentation and see what happens (although I may have to stop short of doing actual controller failovers... users will probably not take too kindly to that).

[Robert Brooks_1]

There does not appear to be a way of telling VMS to re-evaluate the paths and choose the 'best'

--
Whose definition of best path? Fewest devices using it as a "current path"? Fewest I/O's per unit time? Fewest number of bytes transferred per unit time?

Among the things we wanted to explore with multipath was "concurrent multipath", where I/O could be sent down more than one path at a time.

-- Rob

[EdgarZamora_1]

I've gotten the answer I needed from the storage vendor...

Hi Edgar,

The answer to your question is a resounding NO to ALUA!
Do not enable ALUA. OpenVMS does not really support ALUA. (The feeble attempt by HP to do this on the EVA doesn't count).
Since OpenVMS likes static (stable) paths, switching manually is the best approach. Hosts like OpenVMS (at system boot) like to use paths in the order they are presented from the fabric.
The dance goes something like this (on both Alphas and I64):
Host Adapter FGA0 - Who do I see out there??? Give me some target names!!!
Host Adapter FGB0 - Who do I see out there??? Give me some target names!!!
And so on and so on...

Since the OpenVMS sees the same target names over and over (isn't multipathing beautiful?), it selects the first path it sees and says "you're my primary". It then saves the others as alternate/backup paths.
OpenVMS is very fair minded in that it doesn't discriminate on race, color, creed, national origin, or optimal path. So It'll take the prize behind door # 1.

In the real world, this is not always the correct choice. I would suggest that you determine the best (optimal) paths to use, and then place a small script in your systartup_vms chain to set the paths at system boot. Unless there is something else going on in your environment, these paths should remain stable. (Older versions of OpenVMS like 7.2-1 had awful FC polling algorithms and did some path switching on their own. Versions 7.3 and above seem very stable.)

Hope this helps.