MSA2000 host port setup

[RichardJ]

This question applies to VMWare and Windows 2003 server environments. I know all about MPIO and VMWare's embedded multipathing, thats not the issue.

I have heard conflicting arguments about the setup of the MSA2000fc host ports.

do I set the Host port set to loop or Point to Point?
If loop then do I enable host port interconnects?

If point to point, then are there any other settings I should think about.

thanks

[Uwe Zessin]

In a switch environment, you use point-to-point and DO NOT enable the host port interconnect.
I don't think VMware ESX supports a direct connect (implying loop protocol) to the MSA2000fc.

[Gene Laoyan]

I have an MSA2212fc. The fibrs cables I connected are direct from the MSA controller A port 0 to my FC2242SR port 0. VMware ESX 3.5 see's the vDisk LUN's fine.

The problem if you are trying to multipath is thet the second controller on the MSA 2212fc is not presenting the LUN's simultaneously as controller A. In fact, it does not present the LUN's at all. Hence, if you rely on ESX built in miltipathing, you will be dead in the water. I have been trying to get BOTH controllers to present the same lun simultaneously for a few days now.

Try it out if you have two controllers on your MSA2212fc. Connect controller A to your HBA on your server and make sure that controller has been mapped with it's WWN. Now, theoretically if HP's great advertisement found here: http://h18006.www1.hp.com/storage/disk_storage/msa_diskarrays/san_arrays/msa2000fc/index.html then theoretically controller B should be presenting as well. Now connect Controller B to your fibre HBA. You'll get nothing. Either HP is lying on their web page or they shipped products that don't work as they sell the them to be.

An HP rep is supposed to be here tomorrow morning with what they think will fix it. I'll let you know what happens.

Thanks

[dyno_ace]

Gene,

I have a similar problem with my msa2212fc. We have fully redundant fc cabling with 2 StorageWorks 4/8 switches and 4 DL 360 servers. Each server has dual port Qlogic card with port 0 to switch A, port 1 to switch B.

The MSA is cabled as documented, Ctrl A port 0 and Ctrl B port 1 to switch A, Ctrl A port 1 and Ctrl B port 0 to switch B. This provides a fully redundant path to every DL 360 host running ESX server 3.5.

I created 3 test volumes with global LUNS 0, 1 and 2 on the MSA. Some servers see all the volumes on 1 of the 2 hbas, some see selected volumes on both hbas, but none of the 4 servers sees all 3 volumes on both hbas. It seems almost random, except that it doesn't change with reboots or rescans in Vmware.

The MSA storage manager even indicates the problem in the host volume mapping display screen by showing only 7 of the 8 WWPN numbers and various combinations of "Controller A port 0" or "Controller B port 0" or "Both controller A port 1 and Controller B port 0" for each host WWPN in the table. The way it is cabled, I would expect all hosts to see both controllers.

In addition, some servers see multiple paths under Vmware "manage paths" for some LUNS, but no server sees all volumes and paths. I have tried loop and point to point topologies with no difference. I always leave the MSA host port interconnect disabled as documented for my config.

The cabling has been double checked and all the WWPN numbers for all the devices and ports show up properly when getting port and device status from each of the 4/8 switches, so it seems to be the MSA itself which does not work right. The switches are running the latest v6.1.0d fabric, the vmware is the latest 3.5 update 2.

I would be very interested to see the resolution of your problem, as it sounds quite similar to mine.

John

[RichardJ]

My set up is fully redundant, I have a dual FC2012fc, 2x4/8 SAN Switches interconnecting 3 x DL380G5 VMWare servers.
I have noticed that the VMWare servers only saw 2 paths to the LUNs... is that right?

[Uwe Zessin · Accepted solution]

Yes. The non-owning controller does not present any (not even 'passive')
SCSI LUNs, unlike most other storage arrays on this planet.

[RichardJ]

I will leave this thread open for one more day as I want to see Gene and Dyno's comments.
I think that Ewe has answered everyone's question with his last comment.
My MSA2000 is working 100% if Ewe is correct - which is usually the case...!

[dyno_ace]

Richard,

Thanks for your concern. I am glad your problem is resolved, however, mine persists. I understand from Uwe's explanation that only 1 controller will present a lun to a host in a fully redundant switched configuration. Supposedly, if that controller fails completely, the other controller presents all the luns.

If I accept that this is ok (even though a cable or switch or non-critical controller failure will cause one or more luns to be unavailable to one or more hosts) I still cannot get my setup to function corectly.

All luns are setup as global rw to all hosts in the MSA with controller A owning 2 luns and B owning 1. All 4 of my ESX hosts see at least 1 lun, and but only 2 see all the luns. Some of the hosts see multiple paths available for selected luns, but no host sees all luns and paths.

I guess I can't expect the full multi-paths to show up in vmware since both controllers won't present all the luns. However, I should be able to at least see all the luns on all hosts since they all have connections to both controllers. I can't even get this to work.

I am consulting HP for support now. I won't say that I am shocked and disappointed until I know all the facts, but right now I am very concerned.

John

[RichardJ]

John,

Have you tried using LUN masking (SSP in MSA1500 speak) even though you have no need to so in your case, but you then force the controllers to present the LUN numbers to the desired hosts.

Are you using any Switch zoning? I read today that you should zone from the port and not the node WWN. maybe just disable zoning and see if that helps..?

Rich

[Gene Laoyan]

OK, the HP tech guy came in but couldn't do anything for me. The product was too new and I don't believe storage was his "cup of tea".

Uwe is correct, the controllers only present the LUN's through one controller at a time. This sucks and I have sent an email to our HP rep as well as their hardware support guy about this.

RichardJ, there is NO possible way you are fully redundant if only one controller can present them from the MSA2212fc. Masking may just be a method of fooling yourself. No offense.

The current infrastructure this device was supposed to support is an ESX cluster of two nodes. The design I was targeting was to have the folowing reduncaies...

ESX Node "A" dual port fibre controllers
ESX Node "B" dual port fibre controllers
MSA2212fc dual fibre port controller "A"
MSA2212fc dual fibre port controller "B"

NO FIBRE SWITCH!
Didn't budget for it because it says it supports Active/Active.

This would give complete redundancy from port (from controller) to port (on the HBA of server), in theory.

The way this device is currently designed and I don't want to sound errogant, I do not believe those who say they are "FULLY" redundant. This devices's current config can not give controller redudancy in a true Active/Active" config unless you use a fibre switch. You are probably using the single controller "A" with both connections into your fabric switch and from there you ARE seeing your LUN's. Yes, you will have two LUN's going to the same vDisk/volume but to the same controller. Yes, your LUN will be presented only from one controller at a time. Same goes for controller "B".

Let's take a look at what our plan was. ESX cluster. With such a cluster if a fibre path dies, say one port on the HBA died on the ESX server hosting VM's, the MSA2212fc wil NEVER fail over because it still thinks nothing is wrong because it only cares of it's own hardware, not a valid path. Hell, you can unplug the fibre cables and it still won't failover. The built in multipath in ESX is useless because controller "B" never get's "Ownership" (never presents the LUN) of the vDisk. In this case the entire virtual infrastructure is "DEAD IN THE WATER".

Just try it. Use just one cable. Connect the cable from controller A to your HBA on the server. You get a LUN, unplug the cable and plug it into controller B. NO LUN!

Beatin a dead horse here, I hope you get my point.

[RichardJ]

Gene,
Using SAN switches, I pulled cables, shutdown MSA2000 controllers one at a time and VMWare kept on going.
The only thing we noticed is that VMWare doesn't failover over the paths, so this means that the surviving controller is presenting the WWN's of the dead controller and the LUNs of that controller.
VMWare hosts just keeps on going, but the dead path never fails.
I need to go on a course to understand what is happening.
Any comments anyone (Uwe)?

[Gene Laoyan]

Richard, can you enlighten me on how you "shutdown MSA2000 controllers"? I would like to try it. Make it the "Dummies" version so I don't screw it up.....lol.

Thanks

[RichardJ]

On the MSA2000 web interface, click:
Manage
restart
choose a controller to shutdown.

have a good weekend...

[Uwe Zessin]

> The only thing we noticed is that VMWare doesn't failover over the paths,

I don't have an MSA2000fc to try, but if I understand the architecture correctly, you should have only two paths: Port_0 and Port_1 of the controller which is presenting the volume. You can failover between these paths only.

From what I understand: a controller failover should be transparent. In that case, there can be no 'dead paths'.


Can you supply us with the output of a few commands from the service console before and after a 'controller failover'?

ESX multipathing:
# esxcfg-mpath -l

Status information from the Fibre Channel adapter - for Qlogic it is something like:
# find /proc/scsi/qla2300/[0-9] -ls -exec cat {} \;

Sorry, I don't know what the correct command is if you have an Emulex adapter - I think it uses 'lpfc'.

....
If you could run the command immediately after the failover. Then force I/O to the VMFS:

+----
# for f in /vmfs/volumes/[0-9]*; do echo $f; touch $f/write.tmp; rm -f $f/write.tmp; done
-----

and finally run the commands again, it could be interesting, too, thanks!

[Gene Laoyan]

I was finally pointed to the right direction by our HP Enterprise Account Manager.
By default the MSA2212fc is actually Active/Pasive. To correct this, you have to enable controller interconnect. Follow directions below...
To change the host port interconnect setting:
1. Select Manage > General Config > Host Port Configuration.
2. In the Advanced Options panel, click Change FC Port Interconnect Settings.
The Host Port Configuration panel displays the current interconnect setting.
3. Set Internal Host Port Interconnect to Interconnected (enabled) or Straight-through
(disabled).
The default is Straight-through.
This setting affects all host ports on both controllers.
4. Click Save And Continue.
The main Host Port Configuration page is displayed.

This can be found in page 56 of the document found here: http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01394283/c01394283.pdf

Thanks

[Gene Laoyan]

Uwe, I tested path failures on our current config and ESX's multipathing kept the disk up. When I went to mange path it automatically switched paths. I think this will work for us now.

[Tiago Andrade e Silva]

Gene,
How did you tested path failures?
"interconnect" is only used in direct fiber connection from host to storage controller like your environment and I think it doesn't get any different then before because you still don't see all LUNs (on same vDisk) from both controllers at the same time. At least that's what I understand from page 41 in the same manual: ports 0 and 1 (from different controllers) are internally connected but only used when there's a failure with one controller (states the manual). Am I missing something?

I would like to call your attention to the following quote from Q&A HP web site regarding the MSA2000fc dual controller product:
Q2. What does dual "Active/Active" controllers mean?
A2. It means both controllers are ready to process I/Os and provide redundancy. If a controller fails the second controller can handle the I/O requests until the failed controller is brought back online.

This is not the same two controllers beeing able to "control" or "see" the same volumes at the same time. Not the usual and "normal" active/active storage controller configuration.

Regards,
RC

[Tiago Andrade e Silva]

Uwe,

>you should have only two paths: Port_0 and Port_1 of the controller which is >presenting the volume. You can failover between these paths only.

You're right! For example the paths I see in ESX1 are:
* vmhba1:0 (Controller A, port 0)
* vmhba2:1 (Controller A, port 1)
This is for controller A as "owner" and a dual HBA on host. SCSI target 0 only on HBA1 and SCSI target 1 only HBA2. Switched configuration. Only one path per HBA.

Regards,
RC

[Tom Lyczko]

Would anyone please be willing to comment on how this would apply to our dual controller MSA 2012i (iSCSI) with hardware HBAs (QLogic) and ESX/vSphere 4.0??

Should I look at setting up our 2012i to active/passive so VMware's own embedded multi-pathing will work with this SAN??

HP requires me to buy new hardware iSCSI HBA cards to use our 2012i with ESX 4.0, sigh.

Thank you, Tom

[Jorge Pons]

HI

In my case, I have 2 ESX 4.0 Vsphere and 1 MSA2012fc, in direct connection (loop) but I don´t know where I enable host port interconnection. I dont see the option in SMU.
Some one tell me where enable?

Thanks in advance