ESXi-A5120-Storevirtual Link Aggregation Distribution

[robyaps]

Hello , we have

 - 4 A5120 EI configured in true IRF (with rear 10Gbps CX cables in ring topology)

 - 4 storevirtual 4330

 - ESXi 5.5 virtualization host

We want to know what is the best configuration for best performance/link member distribution using link aggregation:

- each 4330 connected to iSCSI VLAN on A5120 with 4 ethernet link

- each ESXi connecet to iSCSi VLAN on A5120 with 4 ethernet link

We know ESXi support LACP, but only using IP Hash distribution algorithm. 4330 uses virtual IP, only one IP for all cluster nodes.

We know 4330 fully support LACP with different distribution algorithm (mac address, ...).

In this scenario:

1. What is the best configuration of Link Aggregation between ESXi and A5120 to have high performance and fair link usage ?

2. What is the best configuration of Link Aggregation between 4330 and A5120 to have high performance and fair link usage ?

We hope someone can show us the best solution and if possible links to documents that might clarify performance based on the type and configuration information to properly determine the distribution on the links members of the link aggregation .

Thank you so much

P.S. This thread has been moved from Switches, Hubs, Modems (Legacy ITRC forum) to Comware-Based. -HP Forum Moderator

[robyaps]

OK. After reading a bit of documentation, we arrived at these conclusions, at least not until someone says otherwise...

We made a cluster of 4 HP StoreVirtual 4330 configured in RAID 10 as suggested. Each 4330 has 4 network card all used in an iscsi VLAN bonded together to maximize inter node communications and storage access performance. This is questionable but for now we decided to try so.

Below network configuration

IMPORTANT: for ESXi and how it works, at least until version 5.5, we can speak of IP Hash only for connections of Virtual Machine, management, vMotion, .... For iSCSI connections, if you follow the best practices Lefthand / ESXi, even if you set the Link Aggregation 802.3ad, the vmkernel port groups will not use IP hash distribution algorithm, but a Multipathingmechanism with Round-Robin distribution over links (RR was recommended on best practices for lefthand/Vmware).

To connect Lefthand - IRF, we used a setup 802.3ad dynamic link aggregation (LACP). We will see if the performance need optimizations or a configuration like ALB, though with the IRF that is a real stack, ALB could create problems with MAC addresses management. 

To connect ESXi - IRF, as we have vSS (vSwitch Standard and not Distributed - we have Essential Plus license...), we configured a static link aggregation on IRF. On ESXi we created VMKernel groups as indicated in the documentation:

- “http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vsphere.storage.doc_50%2FGUID-0D3112...”

- “http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vsphere.storage.doc_50%2FGUID-37F97D..."

- “http://pubs.vmware.com/vsphere-50/index.jsp?topic=%2Fcom.vmware.vsphere.storage.doc_50%2FGUID-C1358D..." 

- “http://h20195.www2.hp.com/V2/GetPDF.aspx%2F4AA3-6918ENW.pdf”

In this way for Port Group VMKernel used for iSCSI connections, ESXi does not use IP HASH distribution Algorithm, but instead Multipathing distribution MRU or RR or Fixed. Solving the complexity of set specific IP and control IP HASH so obtain traffic distribution on links.

We activates VIP-LB on LUN in Lefthand cluster (default choice). So... in theory... we achive a correct setup with a good fair use of links.

any comment will be appreciated.

[Apachez-]

Did you verify through SNMP or such that both physical links are somewhat equal used after you performed the above setup?

[robyaps]

I have to correct an inaccuracy...

...

"To connect ESXi - IRF, as we have vSS (vSwitch Standard and not Distributed - we have Essential Plus license...), we configured a static link aggregation on IRF"

...

This sentence is not correct. Among the interfaces for iscsi ESXi multipathing and IRF you should not create any link aggregation. The ESXi vmk/vmnic interfaces used for multipath will be simple connected on VLAN port for VLAN on the IRF we have dedicated to the iSCSI network. We do not use in this configuration nor VLAN trunk on IRF nor VLAN tag on ESXi.

The system will be in production after a few tests on links and configurations using test VMs. As there is not still sufficient data stream we don't know whether the distribution will be fair on multipath links. For now, the setup works without problems.