Flex 10 LAG algoithm

[jkralles]

Does anyone know if there is a CLI to the Flex-10 switches that allows me to chagne the algorithm it uses to select which switch port is choosen for the data to exit the switch. For my application all of my data is exiting the Flex-10 switch on the same on the same 10Gb fiber. All other switches I have used allows me select other algoithms to help stear data to certain ports and help with data flow management.

[ChrisLynch]

Virtual Connect manages the load balance algorithm automatically.  How it load balances traffic is based upon a few different things:

1. Wether or not there is IP or TCP information in the L3 header

2. What the source and destination MAC address is

VC will then create a hash value based on those two.  Keep in mind that if there is just a single source and destination, then LACP will do nothing to load balance frames across available ports.

A lot of our whitepapers are available here: http://h18004.www1.hp.com/products/blades/components/c-class-tech-function.html

[jkralles]

I found some additional information in an HP document HP Virtual Connect for the Cisco Network Administrator, pages 17 and 18.  It somewhat differs from your resonse in that it states it uses Source and Destination IP address plus TCP or UDP ports. Your information could be more current or out of date. Do you know which is really correct? It is disappointing that the Flex-10 switch does not allow the user to cofigure this like typical vendor switches allow. At least knowing what the true algorithms is will help me steer traffic which i definitely need to do as things exist today.

It would truly be very helpful to know what the hash algorithm is so that I can design my system with that algorithm in mind.  I have 80 Gbits of information to manage for which there are only 4 unique TCP/IP addresses and port pairs on one end of the traffic and thus making the communications less random in nature.

If you could help with the algorithm if that is not proprietary that would be great. It sure would save my a lot of time not having to reverse engineer it.

Thanks

[ChrisLynch]

My response does not differ from what is documented in the VC User Guide or Virtual Connect for Cisco Administrators whitepapers.  If there is no IP information in the header, then the load balance algorithm defaults to Source and Destination MAC Address hash calculation.  None of this is proprietary in any way, and is completely based on the 802.3ad standard.

Please keep in mind that LACP at the end of the day is primarily used for fast uplink, or port failover.  It does provide capabilities to load balance traffic across available ports.  However, LACP is completely based on session load.  Meaning, the fewer number of active converstations, the less chance of traffic being hashed to another available port.

Virtual Connect manages settings automatically, for the vast majority of customer use cases.  In this particular case, VC automatically detects the L2 and L3 header information, and based on what is present, it will load balance accordingly.

Can you tell me what upstream switches you are connecting to, and and you provide the port and Port-Channel/BridgeAgg configuration?  Typically, the default configuration on the switch side is more than sufficient.

[jkralles]

I will get you some of the other information tomorrow that you requested.

However, I have done some preliminary testing and I can not decern that the hash algorithm is using the port number in the algorithm. It is certainly using the source and destination IP address combination for my TCP traffic.  In my case I sent my traffic from 1.1.11.201 to 1.1.11.217 on ports 4000, 9001 thru 9008, 9873, and 9311 and in all cases the output LACP port stayed the same (I have a 4 port LACP). When I varied the source or destination address and then used the above 11 port numberas in all of those cases, the LACP port stayed constant among the 11 ports but did vary when source and destination address were different. I would have expected the ports to vary when using different TCP port number.

Though maNy people may use LACP for failover the specification definely refers to its load balancing capabilities. Even the HP documentation I cited earlier discusses the load balancing capabilities for distributing frames across physical ports.

[blade-guy]

Since the LACP hash is not working for you could you force balance it using Profiles and uplinks rather than a LACP? Much like the Active/Active scenario in the VC cookbook you could define four uplinks on each module. On module A  DataA1,DataA2,DataA3,DataA4  On module B DataB1, DataB2, DataB3, DataB4  Then for example assign blades 1-4 to Data(A1-B1) Blades 4-8 Data(A2-B2)  etc this would force traffic at the blade level  to balance across the four  active links not ideal but it seems like you would be getting better than what the hash is giving you now.  Net result 40Gb traffic more or less balanced assuming 16 servers.

[jkralles]

That would be my "Plan B".  That would probably work similar to a balanced Load Balancing algorithm. However, I would loose failover of a switch port and that is critical to this system.

Also, I believe there is a bug in the Flex-10 implementation of the Source IP, Destination IP and Port LACP algorithm. My goal is also to have HP verify whether that is true of not and if so fix the LACP implementation to the documented implementation.

My application has 3 blade chassis (48 computers) all comminicating with 4 DL360s that each have dual 10Gb bonded uplinks to a fully non-blocking Brocade switch. The DL360s are fronting a SAN operating at 6GBytes/sec and all of the communications go to the 4 DL360s over the same port. The SAN/fiber channel IOs are evenly balanced. Think of it as 4 high speed read/write file servers fanning out to many computers. It's important for the narrow end of the funnel (4 computers) to operate well balanced.  If I had a lot of random comunications all between 48 comptuers I think things would work themselves out better.

I think by understanding the algorithm that was implemented I can choose which IP address to configure the 4 DL360s which will allow a better distribution of data accross the Flex-10 LACP.

[blade-guy]

if you used an active/active setup on the NIC bond you would still have port failover between the VC two modules. Not as robust as a 4 port LACP as each group could only loose one port or one switch.  Glad to hear it might be a viable plan-B

[ChrisLynch]

John, I sent you an email directly outlinin a resolution to this issue.  When I have more public data to share, I will.

Keep in mind that the NIC ports are already in an Active state.  The NIC team can be configured for TLB, or ALB to better improve throughput from the host.  However, we are talking about UPLINK ports, not downlink ports.  Even going with an Active/Active VC Network design, that still does not mitigate the issue you are facing.