EVA 4400 and vSphere 4: active/active vs. active/passive

[sam bell]

Hi,

I'm currently planning a small vSphere cluster consisting of two hosts attached to an EVA4400 and am trying to separate the line between active/active and active/passive in respect to the EVA4400.

In VMware's document "VMware vSphere FC SAN configuration guide" the following is stated about active/active arrays:

"An active-active storage system allows access to the LUNs simultaneously through all available storage ports without significant performance degradation".

Now I've read in a book about vSphere 4 that most "midrange" arrays support something called "ALUA" - which seems to stand for "Asymmetrical Logical Unit Access" and that those "midrange" arrays usually have an internal interconnect between both controllers used for write cach mirroring and other management purposes.

While reading this I remembered about something called "Implicit LUN transition" I read previously in the EVA user guide and the fact that ESX4 recognized the EVA as an ALUA storage array type.

Further on, the book explains that ALUA is an addition to the SCSI standard that enables a LUN to be presented via its primary path and via an asymmetrical (significantly slower) path via the secondary controller, transferring the data over the interal interconnect.

The book also states that the asymmetrical "nonoptimized" path generally comes at a significant performance degradation and that midrange arrays don't have the internal interconnection bandwith to deliver the same response on both controllers. According to the book this is the case because there is usually a relatively small or higher latency internal interconnect used for cache mirroring that is used for ALUA vs. enterprise arrays that have a very high bandwidth internal model.

The book finally states that without ALUA, on an array with an active/passive LUN ownership model, paths to a LUN are shown as "active", "standby" or "dead" in the vSphere client. Instead, when ALUA is used, all paths are shown as "active" though the assymetrical paths are internally marked as "active non optimzed" and never used for I/O.

If I understand the VMware document quoted in the beginning correctly, a "real" active/active array should allow access to the LUN through all the storage ports at the same time - I however can see in my VMware configuration that there is only one path marked "Active (I/O)" at any time and that this is the one that belongs to the managing controller of the LUN (path selection MRU is used).

So...does this apply to the EVA 4400 as well? Does the "Implicit LUN transition" indicate that the EVA actually is an array with an active/passive LUN ownership model and should I take care that only the path to the managing controller of a LUN is used on the ESX hosts?

Thanks

[sam bell]

Just did some further reading and found some interesting articles about this. The first interesting article is available on the blog of Frank Dennemann:

http://www.frankdenneman.nl/?p=5

It states, what I didn't know before, that active/active arrays can be further subdivded into symmetrical active/active (SAA) arrays and asymmetrical active/active (AAA). This really is the key to understand all this since otherwise you're too confused trying to distinguish between just active/active and active/passive arrays.

According to Frank, with symmetrical active/active arrays IO request can be issued over all paths and every controller in the array can accept and send IO to the LUN. HP XP and upper level HDS are real active/active (SAA) arrays but apparently *not* the EVA - which are asymmetrical active/active arrays instead.

He further describes that in such an asymmetric active/active array like the EVA both controllers are online and both can accept IO, but only one controller is assigned as the preferred (owning) controller of the LUN. That's what I read in the description of "Implicit LUN transition" in the EVA user guide. Let me just quote what Frank describes next:

"The owning controller can issue IO commands directly to the LUN. The non-owning controller, - or to make this text more legible - proxy controller can accept IO commands, but cannot communicate with the LUN. For example, if a read request reaches the array through the proxy controller, it will be forwarded to the owning controller of the LUN."

So, this means that the statements I read in the book seem to be correct - and in theory I should take care that each LUN on the ESX host is issuing I/O only using the path to its managing controller.

It however seems that I don't to have to take care about it manually in vSphere 4:

http://www.yellow-bricks.com/2009/09/29/whats-that-alua-exactly/

The article describes that when using the path policy "MRU" in vSphere 4 the ESX host always knows about the optimized path (the path to the managing controller) and only uses the unoptimzed path if the optimized one fails.

Further descriptions about the whole topic are available here:

http://vknowledge.wordpress.com/2009/10/11/vsphere-hp-eva-and-path-policies/

http://virtualgeek.typepad.com/virtual_geek/2009/09/a-couple-important-alua-and-srm-notes.html

Would be interesting to know what the impact on performace might be when using the unoptimized path though.

[Niels Vejrup Pedersen]

Hello,

Regarding the mirror ports - every write to either controller will always go across the mirror ports since this is where the cache mirroring goes.

If you request and read operation to the non-optimized path this can saturate the mirror ports, and will eventually degrade performance on all traffic.

Another thing to consider is the lun trashing issue - if the io's to a vdisk is constantly over a given percentage (default 60%) - then the ownership will change - and would need reconfiguration on the OS's accessing the device (if they are now aware of optimized/non-optimized paths)


Here is a few guidelines.

1) When creating vdisks on an EVA you will need to assign a managing controller for balancing load on the controllers (controller a/b - failover/failback)

2) Since vSphere already knows the EVA arrays as ALUA i would recommend considering using RoundRobin path selection. When using this, half of your paths will always be active.

[Mark...]

Hi,
Maybe this will help you out:
All EVA's run code called VCS or XCS.
EVA3000/5000 run VCS but there are two versions.
VCS V3.xxx = A/P
VCS V4.xxx = A/A
All other EVA's run XCS code and this is all A/A.
Mark...

[sam bell]

But if I understood correctly even A/A needs to be further subdivided into asymmetrical A/A and sysmmetrical A/A. According to the blog of Frank all EVAs are asymmetrical which also fits in the line with what HP describes in the "Implicit LUN transition" sections of their documentation.

So on the EVAs, only one controller can directly serve a LUN at a given time. But, of course, since ALUA is available, the hosts can also issue I/O to the second controller and it is passed to the disks as well. The I/O however is not passed directly to the disks by the second controller like it would be with symmetrical A/A arrays, but is first proxied to the LUN-owning controller and only then issued to the disks.

I think HP (and other vendors as well) should make these differences more clear in their product descriptions.

@ Niels regarding LUN trashing

You wrote that the OS needs to be reconfigured when the ownership changes and the OS isn't aware of the new optimized path. How could this lead to LUN trashing? It could certainly lead to lower performance if the OS uses the unoptimzed path and I/O needs to get proxied to the new owning controller first. But it should not lead to LUN trashing like it would be the case with active/passive arrays I think (causing the LUN to be *unavailable* over the OS configured path after it moved to the new controller).

[sam bell]

> 1) When creating vdisks on an EVA you will
> need to assign a managing controller for
> balancing load on the controllers
> (controller a/b - failover/failback)

Hmm, I just tested a little with this. So far I only have one LUN configured for the ESX servers. I set the managing controller on the EVA to controller 2 and could see on the ESX servers that the paths to controller 2 were the active I/O issuing paths.

I'm now a little confused because after some minutes the EVA reported a controller mastership change and on on the ESX servers the active (I/O) paths changed accordingly to the new managing controller.

I don't understand this because no path to the proxy controller was used for I/O so there actually is nothing what could have caused the change...Any ideas?

[Niels Vejrup Pedersen]

Hello,

About the lun trashing - it's quite common in vmware environments where different hosts uses different paths - which eventually wil cause the ownership to swing back and forth between the controllers.

In all previous versions this is a manual operation to maintain correct lun access per esx host. In vSphere this is built-in PSP part.

Regards

[Chenchen]

Hi Sam

Base on EVA design philosophy. The EVA A/A only need to load balance on the front end by ensuring that workloads are balanced across controllers and host ports.
The most diffrence between A/P and A/A for EVA that is the A/A allows host access to virtual disks through ports on both controller. So, in a mutilpath environment, the traffic and load maybe will switchs to non-optimal path(non-owning controller) when we using mutilpath software to balances the traffic and load with SST or SQL policy. And then, The non-owning controller may take over related LUN become owning controller according to some rules to reduces overhead at one-hour intervals.

That`s all of my understood.

[Xavi.e.r]

it's over a year since this post was updated. I am not sure if you have reviewed this document

-Configuration best practices for HP StorageWorks Enterprise Virtual Array (EVA) family and VMware vSphere 4

you can find them in here
http://h18006.www1.hp.com/storage/arraywhitepapers.html

In short, it mentions that you can distribute the load on both controllers by pretty much designing the path preference under the LUN presentation in the EVA. Furthermore for vSphere 4 and up, HP recommends setting up Round Robin path policy in the ESX servers. see the document for further information. Be aware that also ESX 4.0 U2 resolves a bug issue where the Round Robin policy would be reset after reboot.