vsphere5 MPIO refuses to use more than 2 interfaces

[HansLoots]

A couple of weeks ago we started using 3 DL385G7's with 8 1GE interfaces each and 2 P4500G2's with 2 10GE interfaces.
For the ISCSI network we've reserved 4 GE interfaces on each server. We've setup MPIO in the way that vmware prescribes, with one ip on each iscsi nic. All iscsi nics are in the same subnet.

The saniq boxes (9.5) see the number of LUN's times 4 connections, as expected. Vmware shows the same thing: 4 paths to each LUN. It's set to RoundRobin path selection.

But what I did NOT expect was that vmware refuses to use the last 2 interfaces. The first 2 ARE used as expected, nicely roundrobin, but not the last 2. VI client shows (almost) no traffic; the switch snmp statistics confirm this.

Now I can imagine that MPIO does not work at all if I've setup something the wrong way, but like this is WEIRD, isn't it?

Anyone ideas where to look next for more information? Did I miss something?

[5y53ng]

Hi Hans,

Any progress on this problem? I'm not sure this will resolve the problem, but there is an additional setting you can configure on your VMware hosts to limit the number of consecutive commands issued on each path. The default is 1000. I have been looking for a recommendation for HP storage systems, but I haven't found one. I'll probably experiement with this setting a little to see if any performance increases are possible.

You can read more here: (look for question 3)

http://virtualgeek.typepad.com/virtual_geek/2009/09/a-multivendor-post-on-using-iscsi-with-vmware-vsphere.html

edit: fixed link

[HansLoots]

Hey,

Nope, not much progress. The reason I stuck to the default value of iops=1000 is that I was too lazy to figure out how I could adapt the (abundant) vsphere4 commandline examples to vsphere5.

Today I decided to bite the bullit for that and came up with this:

~ # esxcli storage nmp psp roundrobin deviceconfig get -d naa.whatever

Byte Limit: 10485760
Device: naa.whatever
IOOperation Limit: 1000
Limit Type: Default
Use Active Unoptimized Paths: false
~ # esxcli storage nmp psp roundrobin deviceconfig set -d naa.whatever --iops=1 --type iops
~ # esxcli storage nmp psp roundrobin deviceconfig get -d naa.naa.whatever
Byte Limit: 10485760
Device: naa.whatever
IOOperation Limit: 1
Limit Type: Iops
Use Active Unoptimized Paths: false

However, still no luck. Of all nics in the iscsi collection (vmnic2, vmnic3, vmnic4 & vmnic5) only vmnic2 & vmnic3 are used. No traffic at all on vmnic4 en vmnic5.

Thx.

[GenghisKhan]

Hello Hans

I think what you are seeing is 100% correct behaviour. You have only 2 P4500G2's so you can only have maximum 2 active paths.

As a test, knock up a test lab with 3 VSAs and a vSphere host with 3 dedicated iSCSI NICs set up for MPIO - I'd bet my bottom dollar that then you would see 3 active paths.

Cheers