Network configuration for Nimble

[joebrug52]

I have 4 esxi hosts (7.x).. each with identical hardware including two dual port NICs. I have dedicated two for "management/vmotion" and two for iSCSI. They attach to a meraki switch stack. Each pair of NICs is split across the switches. i.e Nic1 port 1 goes to Switch A, port 2 goes to Switch B. Nic2 port 1 to Switch A, port 2 to Switch B. I'm looking for best practices on how Nimble recommends setting up vSphere Distributed Switching, etc so I can be sure I have things set up for fault tolerance and load balancing. I noticed a slight imbalance on my iSCSI vmnic traffic. We do have the Nimble Connection Manager installed on the hosts. I show they are operating in Directed mode.. should it be round robin.. ? Perhaps thats the reason for the imbalance. Anyway, just looking for proper set up  

[support_s]

System recommended content:

1. HPE Alletra 6000, Alletra 5000, and HPE Nimble Storage Array OS 6.1.2.600 Release Notes | Special Notes for Array OS 6.1.2.600

2. HPE Alletra 6000, Alletra 5000, and HPE Nimble Storage Array OS 6.1.2.502 Release Notes | Special Notes for Array OS 6.1.2.502

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

[Mahesh202]

Hi joebrug52

The VMware hosts that are connected to Nimble arrays can either use vSwitches or Distributed vSwitches, both will work, from a Nimble array point of view there is no preference between them.
you may also refer to the below link: https://infosight.hpe.com/InfoSight/media/cms/active/public/pubs_VMware_Integration_Guide_6_0_x.pdf

Hope this helps.!!

Regards
Mahesh.

[buzzsubash]

Hello Joebrug52,

If you are seeing the multipathing policy as NIMBLE_PSP_DIRECTED, it is expected. It won;t show "Round robin" explicitly.

NIMBLE_PSP_DIRECTED  essentially refers to Path-Selection-Policy as per Nimble recommendation. 

And can you please explain more about imbalance ? Are you seeing packet drops or slowness? also, are you using jumbo frames?

[joebrug52]

Hello,

yes, correct, NIMBLE_PSP_DIRECTED.

By imbalance, I mean that one vmnic nic (there are two for iscsi) has more traffic than the other. For example, looking at the host performance stats, filtered by Network.. the Real Time usage stats show:

vmnic4: 17,889 Avg KBps

vmnic5: 9,800 Avg KBps

Should those stats be a little closer together or is that expected/ok? 

[buzzsubash]

Thanks for sharing.

I am not sure how ESXi handles it, as far as I am aware there is no additional weightage given for traffic unless the multipathing policy is set to anyother apart from roundrobin/PSP directed (both are similar).

We can take a look at the array ports stats to see if there is any underlying errors/hardware issues, you may have to create a support case.

[joebrug52]

Looking at the Nimble > Monitor > Interfaces page, TG1 and TG2 stats are similar. Eth1 and Eth2 are not. So both TG1 and TG2 are used/Active? Are Eth1 and Eth2 both active as well or is one held for fault tolerance. 

[buzzsubash]

Generally, Eth1/Eth2 are used for management/replication subnet and tg1/tg2 are used for iscsi.

eth2 is usually configured only when a second array exist in same group, but this differs based on initial setup. 

[joebrug52]

So if eth1 goes down, does eth2 take over? If so, thats a good reason to have eth2 set up. If not, if eth1 is down.. do the controllers change?

[buzzsubash]

That is correct, if one port goes down, a controller failover happens. By logic of Nimble OS , a controller with highest number of active connections will stay as active. 

[support_s]

Hello,

Let us know if you were able to resolve the issue.

If you are satisfied with the answers then kindly click the "Accept As Solution" button for the most helpful response so that it is beneficial to all community members.

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".