topic Re: EVA, VMS Blade and preferred paths in Operating System - OpenVMS

EVA, VMS Blade and preferred paths

Steve Reece_3 — Wed, 10 Feb 2010 19:49:32 GMT

Hiya,

I should be asking this of my HP Storage Expert, but he's already home for the evening and I have a nagging doubt in my head.

config:
Three EVA6400s with eight disk shelves each.
BL860c blade servers - two enclosures with three and one with eight.
SAN in between the hosts and the storage managed by others. Storage will be configured using Command View and Continuous Access will be doing replication between two of the EVAs.
OpenVMS 8.3-1H1 on all of the blades.

Question:
With VMS being able to do path switching, do I need/want to include a preferred path in the EVA config or should I just rely on VMS to pick a path and, if I don't like it, choose a different path?

With redundant paths between the controllers and no fibre connection between them, I'm suspecting that we should actually just leave VMS to manage it.

Any ideas please???

Steve

Re: EVA, VMS Blade and preferred paths

The Brit — Wed, 10 Feb 2010 20:55:10 GMT

Steve,
As you can see from the Help for "set preferred_path", this is usually used for MSCP served disks.
You could use this at boot time to mount your disks units using a specific path, you would need to have specific knowledge about the IO characteristics of the volume for your path choices to be even close to optimal.
The alternative would be to let the system select a path at boot time and then, if necessary, modify it later using the "set device /path= /switch". As far as I recall, if a path switch is required, (because the current path in not available) the system will switch to the "next available", not necessarily the best.
Unless you have an extremely IO intensive system with specific "HOT" disks, I would recommend letting the system take care of it.
If you are using Virtual Connect (VC) in your enclosures, then you need to consider your VC SAN configuration to ensure the maximum number of redundant paths.

I would also comment that I am surprised that you are using CA for data replication when Volume Shadowing is both available and superior (assuming that you have the EOE license.) CA on EVA is not really synchronous, even when you specify synchronous. (it is basically asynchronous with a large journal file)

but that is just my opinion, it is of course a personal choice.

HTH

Dave.

Re: EVA, VMS Blade and preferred paths

Steve Reece_3 — Wed, 10 Feb 2010 20:58:55 GMT

Thanks Dave.
Host based shadowing would be great, but I can't do that to DR systems that are a couple of hundred miles away and not clustered (the latency of the disks would be too high I fear.)
Steve

Re: EVA, VMS Blade and preferred paths

The Brit — Wed, 10 Feb 2010 21:05:00 GMT

Agreed!

Just be sure that you understand the limitations of CA on the EVA's! Dont get me wrong, I am a big fan of EVA storage, but I work at a location where there have been "issues".

Dave.

Re: EVA, VMS Blade and preferred paths

Hoff — Wed, 10 Feb 2010 21:07:53 GMT

You're probably planning for this, but I'd leave it alone and start rolling up some T4 data under normal load.

I have a basic SAN load-balancing discussion posted at:

http://labs.hoffmanlabs.com/node/1451

And generic DT advice: test the full CA path before you need the recovery.

Re: EVA, VMS Blade and preferred paths

Steve Reece_3 — Wed, 10 Feb 2010 21:16:27 GMT

Hi Hoff,

I'm planning on collecting it when this new environment goes live, but the ancient VAX systems that we're running the apps on at the moment doesn't have anything like T4 on it and we're fairly constrained on what we can and can't install.

The SAN management contact that we have has experience of EVA, but none of VMS yet is telling me that he must have preferred controllers configured on the EVA for every LUN.

Steve

Re: EVA, VMS Blade and preferred paths

Hoff — Wed, 10 Feb 2010 21:25:42 GMT

T4 is built on MONITOR /RECORD, and that exists on VAX. So if you can't get to use T4, you can at least get a baseline.

Re: EVA, VMS Blade and preferred paths

Steve Reece_3 — Wed, 10 Feb 2010 21:47:19 GMT

Good point! Ta!

Re: EVA, VMS Blade and preferred paths

Robert Gezelter — Wed, 10 Feb 2010 23:49:38 GMT

Steve,

T4 itself is effectively a reporting tool. The CPU-related information can be run directly.

I suggest that you install T4 on one of your Alpha/Itanium systems, and take a careful look at the batch job that collects statistics. It is straightforward to implement the MONTIOR/RECORD, even if the system is significantly behind on revisions.

The resulting data file can then be extracted and used with directly with the T4 analysis tools.

- Bob Gezelter, http://www.rlgsc.com

Re: EVA, VMS Blade and preferred paths

Robert Brooks_1 — Thu, 11 Feb 2010 04:15:15 GMT

As far as I recall, if a path switch is required, (because the current path in not available) the system will switch to the "next available", not necessarily the best.

---

Right -- there is no definition of "best".

There is, however, a definition of "worst". An MSCP path will only be automatically switched to if there are no other local paths.
In that case, as soon as a local path becomes available, multipath will automatically "fail back" to that newly-available local path.

Multipath will first try paths that are "connected" before any "non-connected" paths.
This is an older HSG/HSZ artifact from the dual controller setup, where a LUN would only be "connected" to one of the two controllers at a time. For the EVA's, with
active/active support, it's less of an issue, but I believe that there is still a distinction for a dual-controller setup, with a performance penalty on reads.

-- Rob

Re: EVA, VMS Blade and preferred paths

Robert Atkinson — Thu, 11 Feb 2010 09:12:12 GMT

Hi Steve.

We've recently moved to async-CA between Essex and Lincs. Up until then, we'd come to the conclusion of NOT using SET DEV/PATH, as they get pushed around by VMS anyway.

However, because DR disks in a particular group all have to be delivered from the same controller, we've decided to add SET DEV/PATH from VMS for those disks (don't ask me why - I can't remember!).

Another piece of information you may not be aware of is that although the path may be switch from controller A to B, controller B will not necessarily pick up the load. Instead, controller B passes all requests to controller A, and it continues to do the work. If your EVA is heavily loaded, this can start to show.

Cheers, Rob.

Re: EVA, VMS Blade and preferred paths

Uwe Zessin — Thu, 11 Feb 2010 12:05:27 GMT

> active/active support, it's less of an issue, but I believe that there is still a distinction for a dual-controller setup, with a performance penalty on reads.

Right, because on mid-range arrays into which group the EVA belongs (despite the E-nterprise in its name) a single controller is responsible for all physical disk I/O of a particular virtual disk.

Does that mean that VMS still does not understand ALUA (asymmetric logical unit access)?
If it did, you should set up preferred paths (in reality it means you assign management of a virtual disk to a controller...) on the EVA and all VMS hosts should detect this and limit I/O to one of the 'performance paths'.

Re: EVA, VMS Blade and preferred paths

JBF — Wed, 24 Feb 2010 23:56:46 GMT

Steve,

Since V8.3 OpenVMS understands and uses Asymmetric Logical Unit Access (ALUA) with the EVA storage arrays.

OpenVMS will always choose a path based on which controller manages the vdisk. It also uses this information during a mount verification event. For example, if a path fails (such as when a fibre cable is removed) then OpenVMS will (1) first retry the current path, then (b) search for another Active Optimized (AO) path (on the same controller) to the LUN, and then if necessary choose an Active Not Optimized (ANO) path on any other controller that can access the LUN. If OpenVMS must move to an ANO path, then OpenVMS requests the new controller assume control of the LUN and make it an Active Optimized path.

Due to the coordination between nodes in a VMScluster, I do NOT reocmmend using preferred pathing on the EVA storage array. I have worked several cases where this led to problems. Instead, I recommend allowing OpenVMS to pick a path.

However, on top of that, I suggest you build a DCL command procedure to periodically perform a SET DEVICE/SWICH/PATH=. Ideally you will take the top "n" devices (in terms of the I/O request rate) and spread those across the available paths. This only needs to be done on one node in the VMScluster if you run OpenVMS V8.3 or later. And you probably only need to do this once a day.

Why do this? It helps avoid problems where all the LUNs are switched to one controller (such as when a switch or controller is rebooted). This will at least daily be certain your top workload is balanced across the paths.

Hope that helps!
=jbf=

John B. Fisher

Re: EVA, VMS Blade and preferred paths

Robert Atkinson — Thu, 25 Feb 2010 08:56:05 GMT

Interesting reading John. I'd heard that VMS had become more 'aware' of the controllers, but this is the first information I've seen to back that up.

Could you expand a little more on "Due to the coordination between nodes in a VMScluster, I do NOT reocmmend using preferred pathing on the EVA storage array"

Why is this important, and how would it effect the use of DR disk groups?

Thanks, Rob.

Re: EVA, VMS Blade and preferred paths

Ian Miller. — Thu, 25 Feb 2010 12:07:52 GMT

will multipath polling switch to a AO path from a ANO path or does the polling just check that the path works?

Re: EVA, VMS Blade and preferred paths

Robert Brooks_1 — Fri, 26 Feb 2010 04:59:17 GMT

will multipath polling switch to a AO path from a ANO path or does the polling just check that the path works?

--

The multipath poller is very lightweight. The entity that's relevant is the path, not any one device. If *any* device is reachable on a path, that path is deemed "working".

The details for those with the sources:

Take a look at the mpdev_ppb structure in [lib]
and the code in [sys]mpdev_poller.c. You'll see that the mpdev_ppb structure maintains a list of all devices on a path; thus, there is an mpdev_ppb structure for each path on a system.

The poller walks this list of devices, sending an io$_path_verify I/O. If the first device responds correctly, the poller for that path terminates, and waits for the next interval (usually 60 seconds, if a path is OK). If the first device has "issues", then we'll try the next device. Only after *every* device on a path has failed do we declare the path "bad", and we emit an OPCOM message to that effect.

The poller does not affect current paths for any device *except* for the case of "failback" from an MSCP path. If we've failed over to an MSCP path because all fibre paths are down, it's the poller's job to notice that a device that automatically switched to the MSCP path can now be "failed back" to a fibre path. In that case, the poller will initiate a path switch away from the MSCP path. If, for some reason, the MSCP path was manually selected, the poller will leave it on the MSCP path.

-- Rob

Re: EVA, VMS Blade and preferred paths

Ian Miller. — Fri, 26 Feb 2010 10:13:29 GMT

There does not appear to be a way of telling VMS to re-evaluate the paths and choose the 'best'

Re: EVA, VMS Blade and preferred paths

EdgarZamora_1 — Thu, 02 Jun 2011 15:26:07 GMT

In reading this thread, John (Fisher) mentions that OpenVMS 8.3 is ALUA-aware. I was wondering where I could find documentation on this. Unless I missed it, I didn't find it anywhere in the OpenVMS docs.

I guess I will do some actual experimentation and see what happens (although I may have to stop short of doing actual controller failovers... users will probably not take too kindly to that).

Re: EVA, VMS Blade and preferred paths

Robert Brooks_1 — Thu, 02 Jun 2011 17:15:30 GMT

There does not appear to be a way of telling VMS to re-evaluate the paths and choose the 'best'

--
Whose definition of best path? Fewest devices using it as a "current path"? Fewest I/O's per unit time? Fewest number of bytes transferred per unit time?

Among the things we wanted to explore with multipath was "concurrent multipath", where I/O could be sent down more than one path at a time.

-- Rob

Re: EVA, VMS Blade and preferred paths

EdgarZamora_1 — Thu, 02 Jun 2011 17:24:56 GMT

I've gotten the answer I needed from the storage vendor...

Hi Edgar,

The answer to your question is a resounding NO to ALUA!
Do not enable ALUA. OpenVMS does not really support ALUA. (The feeble attempt by HP to do this on the EVA doesn't count).
Since OpenVMS likes static (stable) paths, switching manually is the best approach. Hosts like OpenVMS (at system boot) like to use paths in the order they are presented from the fabric.
The dance goes something like this (on both Alphas and I64):
Host Adapter FGA0 - Who do I see out there??? Give me some target names!!!
Host Adapter FGB0 - Who do I see out there??? Give me some target names!!!
And so on and so on...

Since the OpenVMS sees the same target names over and over (isn't multipathing beautiful?), it selects the first path it sees and says "you're my primary". It then saves the others as alternate/backup paths.
OpenVMS is very fair minded in that it doesn't discriminate on race, color, creed, national origin, or optimal path. So It'll take the prize behind door # 1.

In the real world, this is not always the correct choice. I would suggest that you determine the best (optimal) paths to use, and then place a small script in your systartup_vms chain to set the paths at system boot. Unless there is something else going on in your environment, these paths should remain stable. (Older versions of OpenVMS like 7.2-1 had awful FC polling algorithms and did some path switching on their own. Versions 7.3 and above seem very stable.)

Hope this helps.