Problems with setting up basic LACP between VMWare ESXi 5.5 vswitch and 4512zl switchports

[qs-it]

I have been attempting to create a basic two-port LAG, between by HP 4512zl switch, and two 10GbE interfaces on a VMWare ESXi 5.5 host, and no matter what combination on the switch I attempt, the LAG never comes up.

I've got the ESXi server set for IP hash based load balancing, and it's using the default VLAN 1 tag, since that's the VLAN that the server and it's guest VMs will be passing traffic on.

I then configure the two switchports that are uplinked to the ESXi server, using the command:

trunk PortA1,PortB1 Trk(#) Trunk

The trunk then get created but the switch then never negotiates LACP between itself and the ESXi server.  I attempted to configure the trunk as

trunk PortA1,B1 Trk(#) lacp

But in that instance, the switch shows that the partner LACP channel never gets acknowledged.

I've performed this task a dozen times on Cisco's with EtherChannel but this is the first time with an HP and I'm shocked that it's been so difficult.  

Any ideas what I should look for?

P.S. This thread has been moved from Switches, Hubs, Modems (Legacy ITRC forum) to ProCurve / ProVision-Based. - Hp Forum moderator

[EricAtHP]

ESXi only supports LACP on Distributed Switches. You need a non-protocol link aggregation. Use the command: 

HP-5412zl(config)# trunk A1,B1 trk1 trunk

[qs-it]

I've tried using 'trunk A1,B1 trk1 trunk' to form the channel, but as soon as I do traffic stops flowing to the ESXi server.

Right now, I've got the two NICs on the VMWare server configured in just standard vswitch0.   Are you saying that if I want to do LACP, I will need to set up a distributed vswitch in VMWare and then enable LACP from there?

Do you happen to know what the default load-balancing method VMWare is using, if the two NICs are configured for load balancing in just a basic vswitch?  Are they doing round-robin load balancing if I'm using IP hash?

[EricAtHP]

Yes, LACP is only supported on distributed switches. Here are some links to VMwares docs that may help:

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1001938

http://kb.vmware.com/selfservice/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1004048

In the past, I have seen some issues if you try to manage an esxi host over a link aggregation. In other words, you might try keeping your Management Network(VMKernel Port) on the esxi host on a single connection and then use a link aggregation for your Virtual Machine Port Groups / VMs.

Also, you do have to change the Load Balancing setting in ESXi to "Route based on IP hash". Go tothe vSwitch properties, edit the vSwitch, go to the NIC Teaming tab, and edit the Load Balancing setting.

[EricAtHP]

I figured out the issue with the management network. On some versions of esxi it appears that the management network doesn't inherit the properties of the associated vswitch by default yet the VM network does inherit as expected. So you either need to edit both the vswitch and the management network for "Route based on IP hash" or edit the management network to inherit the vswitch settings.

[qs-it]

Thanks, Eric.

It also appears that every since version 5.1, ESXi doesn't support LACP in anything but their enterprise level licensing.

Which means that even if I put both interfaces into a single vswitch, and set it for IP hash load-balancing, it looks like it's only going to be doing port interface load balancing and not true LACP.

[Vince-Whirlwind]

I think you need to be careful about using the expression "load balancing" in relation to LACP - some people can be misled by this into thinking that LACP actually provides genuine load balancing.

I can't say I have had any problems with LACP between HP Procurve 5412s and VMWare, EXCEPT for the hash method - the default method seems to have a bug in it which results in massive packet loss and has to be changed to something else. I think we went with the next option down the drop-down list.

[qs-it]

Hi Eric,

So it appears that the out-of-box load balancing that VMWare ESXi does, is only port-based load balancing (not sure what algorithm they use for it.)   I've got 'Route based on IP hash" configured for the NICs grouped in the vswitch that's carrying traffic between the ESXi server and my 4512zl switch, but the switchports on the HP aren't configured as anything but the default.   

I haven't seen a way to configure a static link aggregation between the ESXi server, so that multiple ports can properly load balance the traffic, and increase my aggregate network throughput to the ESXi server.

IS there a way to do static aggregation (without LACP) from the HP switch?

[qs-it]

According to the VMWare documentation

"if 'route based on IP hash' is selected in NIC Teaming, the switch the interfaces are connected to must be 802.3ad compliant and configured to use that link-aggregation standard in static mode."

So I guess the question then changes to how do I configure 802.3ad compliant link-aggregation on a 4512zl, across multiple switchports?   Or is that even a supported configurable option on this particuarly model switch?

[EricAtHP]

Yes, that is the command we talked about above: "HP-5412zl(config)# trunk A1,B1 trk1 trunk". That along with "Route based on IP hash" on all of the ESXi vSwitch componets will create a LAG between the switch and the hypervisor. Both will use their own algorithms to spread the traffic accross the links.

Keep in mind that LAG's are not guaranteed to load balance. All traffic for a converstion will go over a single link. It is only with a true random group of unique source & destination IPs that you will start to see both links utilized. 

[Vince-Whirlwind]

Some vendors may provide the ability to choose an optional port-based algorithm, but as a general rule, link aggregation (whether it is 802.3ad/LACP or something else, like Etherchannel) uses an algorithm based on layer2 or layer3 characteristics on a per-frame basis.

Best to call it "link aggregation", which is what it is, rather than "load balacning" which it really isn't.

[qs-it]

Hi Eric,

That was the issue!   I hadn't set Route Based on IP Hash, at the vswitch level (only the network level) and therefore it wasn't working.   Once I set it at the vswitch level, and allows that to properly propagate to the VM Network and Management Network, I got

Is there a command where I can see what the load-balancing and traffic flow counters are for a particular trunk?   Will the switch show me the aggregated bandwidth of the two ports that I now have in trunking mode with my ESXi server?

[Vince-Whirlwind]

try "show trunk" and "show trunk trk1"

It aggregates the ports, not the bandwidth. Your maximum throughput for any single communication flow is the bandwidth of a single member of the trunk.