Adding a third node to a cluster causes one of five datastores to fail in the VMware environment

[efctns]

Hi,

short pre-story:

we had a cluster of two nodes and wanted to add a third node to be able to increase the volume size of the existing volumes. So we added a new LeftHand 12.6 Node to the existing 11.5 cluster. My understanding here is that you can add a node with a higher LeftHand OS verson to a cluster with a lower version, the HP support confirmed that.

So we added the third node to the cluster and the system started its restripe. The second we added the node though, there was an error that the iSCSI IP of that node could not be reached by any of the VIPs (we have only one). The restripe continued though and pings from that node to the VIP were successful. Pings with mtu set to 9000 to our three ESX hosts were successful too.

Now after approx. 30 minutes one of the five datastores lost its connection. As soon as we power off the newly added node the datastore is online again.

I already have a support case open but the HP support cannot find any problems nor a solution.

We checked the settings on swtiches, ESX hosts, the node and the cluster several times but we cannot seem to find anything wrong.

The configuration looks something like this:

Nodes have a 10GBe bond and a 1GBe bond. 10GBe is iSCSI and 1GBe is management. We have two Core switches (also HP) in HA with one of the 10GBe cables going into one core, the other goes into the other core. Same for the management bond and same for the 10GBe iSCSI of the ESX hosts. iSCSI network and management network are in a seperate VLAN each. Jumbo frames are active in the iSCSI VLAN and on all iSCSI nics of the ESX hosts and the vSwitch.

The ESX 5.5 hosts have a seperate vSwitch for the two iSCSI nics which have an IP-adress in the iSCSI Network each. Standby adapter is only configured on the vSwitch.

Support Case is 5314437093

Did someone maybe had a similar problem and can give any advice ?

Thanks in advance !

Kind regards,

Eric

[Stor_Mort]

It looks like HPE Engineering sees an iSCSI redirect problem with the hardware iSCSI initiator.