topic Query: Network configuration for Nimble in HPE Alletra Storage

Network configuration for Nimble

joebrug52 — Wed, 19 Feb 2025 13:57:13 GMT

I have 4 esxi hosts (7.x).. each with identical hardware including two dual port NICs. I have dedicated two for "management/vmotion" and two for iSCSI. They attach to a meraki switch stack. Each pair of NICs is split across the switches. i.e Nic1 port 1 goes to Switch A, port 2 goes to Switch B. Nic2 port 1 to Switch A, port 2 to Switch B. I'm looking for best practices on how Nimble recommends setting up vSphere Distributed Switching, etc so I can be sure I have things set up for fault tolerance and load balancing. I noticed a slight imbalance on my iSCSI vmnic traffic. We do have the Nimble Connection Manager installed on the hosts. I show they are operating in Directed mode.. should it be round robin.. ? Perhaps thats the reason for the imbalance. Anyway, just looking for proper set up

Query: Network configuration for Nimble

support_s — Wed, 19 Feb 2025 04:59:21 GMT

System recommended content:

1. HPE Alletra 6000, Alletra 5000, and HPE Nimble Storage Array OS 6.1.2.600 Release Notes | Special Notes for Array OS 6.1.2.600

2. HPE Alletra 6000, Alletra 5000, and HPE Nimble Storage Array OS 6.1.2.502 Release Notes | Special Notes for Array OS 6.1.2.502

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".

Thank you for being a HPE valuable community member.

Re: Network configuration for Nimble

Mahesh202 — Wed, 19 Feb 2025 05:31:33 GMT

Hi joebrug52

The VMware hosts that are connected to Nimble arrays can either use vSwitches or Distributed vSwitches, both will work, from a Nimble array point of view there is no preference between them.
you may also refer to the below link: https://infosight.hpe.com/InfoSight/media/cms/active/public/pubs_VMware_Integration_Guide_6_0_x.pdf

Hope this helps.!!

Regards
Mahesh.

Re: Network configuration for Nimble

buzzsubash — Wed, 19 Feb 2025 05:53:18 GMT

Hello Joebrug52,

If you are seeing the multipathing policy as NIMBLE_PSP_DIRECTED, it is expected. It won;t show "Round robin" explicitly.

NIMBLE_PSP_DIRECTED essentially refers to Path-Selection-Policy as per Nimble recommendation.

And can you please explain more about imbalance ? Are you seeing packet drops or slowness? also, are you using jumbo frames?

Re: Network configuration for Nimble

joebrug52 — Wed, 19 Feb 2025 08:43:16 GMT

Hello,

yes, correct, NIMBLE_PSP_DIRECTED.

By imbalance, I mean that one vmnic nic (there are two for iscsi) has more traffic than the other. For example, looking at the host performance stats, filtered by Network.. the Real Time usage stats show:

vmnic4: 17,889 Avg KBps

vmnic5: 9,800 Avg KBps

Should those stats be a little closer together or is that expected/ok?

Re: Network configuration for Nimble

buzzsubash — Wed, 19 Feb 2025 08:52:54 GMT

Thanks for sharing.

I am not sure how ESXi handles it, as far as I am aware there is no additional weightage given for traffic unless the multipathing policy is set to anyother apart from roundrobin/PSP directed (both are similar).

We can take a look at the array ports stats to see if there is any underlying errors/hardware issues, you may have to create a support case.

Re: Network configuration for Nimble

joebrug52 — Wed, 19 Feb 2025 09:12:11 GMT

Looking at the Nimble > Monitor > Interfaces page, TG1 and TG2 stats are similar. Eth1 and Eth2 are not. So both TG1 and TG2 are used/Active? Are Eth1 and Eth2 both active as well or is one held for fault tolerance.

Re: Network configuration for Nimble

buzzsubash — Wed, 19 Feb 2025 11:06:36 GMT

Generally, Eth1/Eth2 are used for management/replication subnet and tg1/tg2 are used for iscsi.

eth2 is usually configured only when a second array exist in same group, but this differs based on initial setup.

Re: Network configuration for Nimble

joebrug52 — Wed, 19 Feb 2025 18:12:11 GMT

So if eth1 goes down, does eth2 take over? If so, thats a good reason to have eth2 set up. If not, if eth1 is down.. do the controllers change?

Re: Network configuration for Nimble

buzzsubash — Fri, 21 Feb 2025 01:06:21 GMT

That is correct, if one port goes down, a controller failover happens. By logic of Nimble OS , a controller with highest number of active connections will stay as active.

Query: Network configuration for Nimble

support_s — Mon, 24 Feb 2025 14:15:04 GMT

Hello,

Let us know if you were able to resolve the issue.

If you are satisfied with the answers then kindly click the "Accept As Solution" button for the most helpful response so that it is beneficial to all community members.

Please click on "Thumbs Up/Kudo" icon to give a "Kudo".