Cluster Services prevents basic MPIO Failover

[Jeff Tam]

Hello,
I am currently working on a project to where we have (2) HP Proliant DL 360 G5 and an HP MSA 2012i SAN module. The setup is of iSCSI and we are leveraging the systems for a Microsoft Windows 2003 x64 Enterprise Edition cluster and SQL 2005 x64 Enterprise Edition cluster as well. The servers each have a total of 4 separate NICs (public, private, iSCSI 0, and iSCSI 1). I was able to get the iSCSI setup working just fine and the servers are able to view the mapped volumes from the MSA 2012i as shared disks. When we moved forward with the Windows clustering setup, we were able to get everything working as well. However, we are seeing a problem during our failover testing. As noted above, the iSCSI 0 and iSCSI 1 ports are each connected to the SAN through an independent network switch. If I power off the active switch (for iSCSI 0), the iSCSI connections never failover to the secondary iSCSI 1 port on the servers. We tested this prior to configuring Cluster services and everything failed over correctly. If I stop the Cluster service, the connections become active on iSCSI 1 again. I searched everywhere and there was some information regarding the "PersistentReservation" registry hack but that didn't work for us. Has anyone got any ideas on recommendations or solutions to this problem other than having to call Microsoft for a support case?

[Bruno J. Melo]

Hi Jeff,

Have you installed the MPIO driver on the hosts?

Read the HP StorageWorks MSA2000 Family MPIO DSM installation guide (485499-002, May 2008):

http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual〈=en&cc=us&docIndexId=64179&taskId=101&prodTypeId=12169&prodSeriesId=3687128

There's also a new firmware version available that may help:

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=3687132&prodTypeId=12169&prodSeriesId=3687128&swLang=13&taskId=135&swEnvOID=1060

[Bruno J. Melo]

Please note:

"If you are using Windows Server 2003 and your array is the 2012i Modular
Smart Array (iSCSI), the required MPIO DSM is included in the Microsoft
iSCSI Software Initiator." on page 5.

[Jeff Tam]

Yes. I have already installed the Microsoft iSCSI Initiator and set up the proper targets and connections. Basic MPIO failover works just fine up to the point when I configure Microsoft Clustering Services. If clustering service is running, connectivity to the SAN does not failover to the secondary iSCSI NIC (verified by running a continuous ping test to the SAN host port). The moment that I stop the Clustering service in Windows, my ping tests to the SAN generates replies.

[Bruno J. Melo]

Hi again,

Can you detail your configuration a little more?
Which NICs are you using for the iSCSI network? Are you using NIC teaming on any of these NICs?
Windows Service Pack?
iSCSI initiator version?
iSCSI initiator configuration?
MSA2012i firmware version?

[Jeff Tam]

Hi Brian,

Yes. Here is some additional information regarding our configuration.

(2) HP Proliant DL 360 G5 Servers
Windows 2003 Enterprise R2 x64 w/ Service Pack 2.
Microsoft iSCSI Software Initiator v 2.06
(1) HP MSA 2012i SAN module
(2) HP Procurve 2810 switches for iSCSI
(2) HP NC380T PCI E Dual Port Multifunction Gigabit Server Adapters for iSCSI

On Server1, we have configured the iSCSI ports as follows: port 0 = 10.251.125.201 and port 1 = 10.251.125.202. On Server 2, we have configured the iSCSI ports as follows: port 0 = 10.251.125.203 and port 1 = 10.251.125.204.

On the MSA 2012i (dual controller), we have configured the iSCSI target ports as follows: Controller A port 0 = 10.251.125.101 and port 1 = 10.251.125.102. Controller B port 0 = 10.251.125.103 and port 1 = 10.251.125.104.

Ports 0 on the servers and MSA are linked on one of the Procurve switches. Ports 1 on the servers and MSA are linked to the secondary Procurve switch. The switches are not interlinked. Without Microsoft Clustering Services configured on either of the servers, the MPIO basic failover works as expected. For instance, on Server1 I run a successful continuous ping test to 10.251.125.101 and 10.251.125.103 through iSCSI switch 0. 10.251.125.102 and 10.251.125.104 do not respond during this time. If I shut off iSCSI switch 0, my ping tests to 10.251.125.102 and 10.251.125.104 then start replying through iSCSI switch 1 so this tells me the MPIO failsover. However, once I configure Microsoft Clustering Service, the same tests as just described no longer failover and 10.251.125.102 and 10.251.125.104 never reply. My fear is that the NICs we purchased do not support MPIO / MSCS?

Any ideas or recommendation? I noticed that Microsoft just released iSCSI Software Initiator version 2.07.


Thanks,


Jeff

[Jeff Tam]

Sorry about the name confusion. I meant Bruno.


Jeff

[Bruno J. Melo]

Hi again Jeff,

You didn't describe your iSCSI initiator configuration.
Microsoft MPIO translates commands to SPC-3 (SCSI Primary Commands 3) which is the latest standard, instead of SPC-2 that is used on Windows Server 2003 Clustering Services.
You'll need a SPC-3 capable iSCSI target in order for this to work.
Check if your MSA is loaded with the latest firmware and if it is capable of SPC-3.
You also need to use persistent bindings your iSCSI initiator...

HTH,

BJM