Multipath trouble with VMware ESX 4.0 and P2000 G3 SAS with SC08e HBAs

[Luca Maranzano]

We bought a P2000 G3 SAS (dual controller) and we have started the first installation and configuration with a DL 385 G6 with 2 SC08E HBAs, to have a fully redundant dual-path configuration.

The cabling layout is the following:

Server1 HBA1 - Port1E --> ControllerA Port2
Server1 HBA2 - Port1E --> ControllerB Port2

The server is running VMware ESX 4.0 Update 2 with the latest updated driver downloaded from VMware site:
vmware-esx-drivers-scsi-mpt2sas-400.08.00.00.00.1vmw-1vmw.2.17.00000

We also run the Firware Maintenance CD version 9.20B on the DL 385G6 server, to be sure to have the latest firmware.

The P2000 is running the latest Firmware version T200R21.

We have good experience with FC and iSCSI connectivity, while this is our first SAS installation, and I was supposing it should be simpler and straightforward than FC :-)

We have created a 1TB Volume mapped to the 2 HBAs on both P2000 controllers, with Explicit Mapping.
The problem is that the LUN is correctly visible on one path, but the other path is seen as DEAD.
The Enclosure on LUN ID=0 is seen on both HBAs.

ESX in /var/log/vmkwarning is reporting something like this:
VMW_SATP_LOCAL: satp_local_claim: VMW_SATP_LOCAL does not support multiple paths per device. Refusing to claim path vmhba3:C0:T0:L1

From the Manage Paths we see a Storage Array Type as VMW_SATP_LOCAL with a default Fixed path selection policy.

Here http://www.vmware.com/resources/compatibility/detail.php?device_cat=san&device_id=17132
the suggested policy seems to be "Vendor_Unique" but I don't know the meaning.

We've searched around for some documentation but we didn't find anything useful.

Is there some particular configuration to do in order to achieve multipath ?

Thanks in advance.
Kind regards,
Luca

[Antonio Milanese · Accepted solution]

Hi Luca,

you have to manually add SATP rules because default ones doesnt include P2000 and so it's reverted to VMW_SATP_LOCAL:

P2000 is a ALUA capable so log on esxi host and add a new satp rule

esxcli nmp satp addrule -satp="VMW_SATP_ALUA" -vendor="HP" -model="P2000" â description="HP P2000 with ALUA support" â claim-option tpgs_on

and then add a default RR PSP policy

esxcli nmp satp setdefaultpsp -psp VMW_PSP_RR -satp VMW_SATP_ALUA

or, since you have already presented luns

for i in `ls /vmfs/devices/disks/ | grep naa.600` ; do echo $i ; esxcli nmp roundrobin setconfig â type "iops" â iops=1000 â device $i ;done

Check that all it's fine:

esxcli nmp satp list

For enclosure LUN0 it's advisable to mask it or change host profile to "OpenVMS" and so it's not presented at all.


Regards,

Antonio

[Luca Maranzano]

Hello Antonio,
thank you for your hint.
Please note that for me your command did not work out of the box.
I had to use a regular expression in the Model string since the exact model is "P2000 G3 SAS", so the command becomes:

esxcli nmp satp addrule --satp="VMW_SATP_ALUA" --vendor="HP" --model="^P2000*" --description="HP P2000 with ALUA support" --claim-option=tpgs_on

One strage thing is that the /var/log/vmkernel contains a lot of messages like this:

May 3 13:22:58 myesx vmkernel: 13:15:45:35.554 cpu9:4105)<4>mpt2sas1: log_info(0x30030101): originator(IOP), code(0x03), sub_code(0x0101)

and I cannot say it they are harmful or not.

I've made some searches around, and found this VMware KB:
http://kb.vmware.com/kb/1003327

but it seems that it does not apply to ESX 4.x.

BTW: the SC08e card during the POST reports firmware 7.05.04 and I didn't find any link for updating it.

Thanks,
Luca

[Antonio Milanese]

Hi Luca,

have you masked the enclosure lun0?
if not you SHOULD..if yes well IOCSTATUS and derived log_info(0x30030101) suggest an invalid register page during periodic ctrl rescan/refresh..i donno if it's a driver problem but several times before linux mptX drivers (vmware ones AFAIK are derived from the same codebase) have had similar issues but those warnings were altogether harmless.
There are other surrounding log_info..did LUNs/paths timeout or were detached/attached during normal operations?

Regards,
Antonio

[Luca Maranzano]

Hi Antonio,

yes I masked the Enclosure Lun0 setting the Host Profile to OpenVMS.

I've performed some stress test with IOmeter and I got good performance numbers.
Till now no other issues apart these log_info messages.

I'll open a SR to VMware support about this.

Thank you.
Luca