ProCurve 2900 trunking between servers

[Andrew Boyer]

Hello,
I am attempting to set up a ProCurve 2900-48G to work with a small cluster of servers. In this test, the two servers are set up with 8 GbE ports each. Each server also has 8 IP addresses.

If I disable trunking, the servers assign one IP to each port, spread the data among the IP addresses, and utilize all eight connections.

If I enable trunking (8 ports per trunk), however, I see two problems. The first - that each server only uses four of its interfaces to send packets - is obviously a function of our OS and not directly related to the ProCurve.

The second problem is that the four packet flows coming from server A get combined by the switch onto the lowest-numbered interface to server B, and vice versa. We end up completely throttled to the bandwidth of a single interface.

This is not the behavior I was expecting - the switch should direct each input flow from server A onto one of the output ports to server B, randomly but consistently. The Src and Dst MAC addresses are the same on all packets because of the link aggregation, but the packets are still spread among the 8 IP addresses.

Trunking / Static LACP / Dynamic LACP don't seem to make any difference.

Is there a setting that I have overlooked? We aren't running the latest firmware at the moment (T.12.51), but I read through the release notes and didn't find anything related to this problem.

Thank you in advance.

-Andrew

[Andrew Boyer]

Update: neither flow control nor firmware T.13.71 made any difference.

[Lmm_1]

The load balancing algorithm to send traffic is determined by the server, if its an HP server runnig windows you can use the Network configuration utility, it will allow you to choose the load balancing algortithm (MAC, IP, Port, RR). Having said that, you should be able to use all 8 ports to send traffic if using an static trunk or LACP.

On the other hand, Procurve switches use Src-Dest MAC as their Trunking load balancing algorithm and is not a configurable setting. (You can change that in other vendors like Cisco). The real adavantage of using trunking for load balancing will be when having multiple conversations with multiple hosts, in this case, on the same subnet.

[rick jones]

Strictly speaking, if something is operating purely as a switch (aka a layer 2 device) it should not look at anything beyond layer 2 information (eg the src and dst MAC addresses. That of course is the "purist's" point of view.

Which OS/rev are you using? For example Linux bonding "ALB" mode does clever things with MAC addresses in ARP replies that may help get things spread-out (or may not).

Near term, if the small cluster of servers is indeed going to be only two nodes, then you might consider connecting the systems back-to-back and avoid the 2900 entirely (as aghast as this might make any ProCurve folks reading :) If the cluster is to grow beyond two nodes, you might consider 10Gig rather than trunked GbE - especially if you are going to be trunking 8 GbE links at one time.

[Andrew Boyer]

Thank you for the feedback. I'll be able to customize the OS (open source) to do whatever I need it to, but I'm still concerned about the ProCurve's hashing algorithm. You both state that it depends on the Src/Dst MAC addresses.

The "Management and Configuration Guide" page 11-28 (T.13.01, January 2008) says:
"The actual distribution of the traffic through a trunk depends on a calculation using bits from the Source Address and Destination address. When an IP address is available, the calculation includes the last five bits of the IP source address and IP destination address, otherwise the MAC addresses are used."

Is the documentation wrong, or is there another factor at work here? I know these switches contain a lot of higher-level routing features that we aren't using. Do I need to turn on L3 features to get the switch to look at the IP addresses?

The two-node cluster is just a test case so that I can more easily monitor the traffic. Our actual deployments will be more like 3, 6, or 12 nodes. We also ship 10GbE systems, but a lot of our customers haven't upgraded yet.

[rick jones]

I've only ever used switches as switches, not as routers, but since this is a test environment, enabling L3 features might be an interesting experiment to see if it affects whether IP addresses get used. what are the last octets of all the IP addresses being used in your testing?

[Andrew Boyer]

I had to give the hardware back for a while, but I'll try to enable some advanced functions when I get another shot at it.

The IPs were sequential, .38-.45 on server A and .46-.53 on server B.