Troubleshooting LACP on switch side ?

[norbertsch]

Hello,

we have a HP 2510-24G and 18 realservers (1Gig), all are running 2 virtual servers. so it is 36 clients.

All clients are connecting another realserver. The Realserver has 2 * 1Gig network to the HP 2510-24G, configured with lacp.

The problem... alls clients are using the same channel to the realserver, so i have only 1Gig throuput.

Why ?

Configuration:

show trunk

 Load Balancing

  Port | Name                             Type      | Group Type
  ---- + -------------------------------- --------- + ----- -----
  19   |                                  100/1000T | Trk1  LACP
  22   |                                  100/1000T | Trk1  LACP

show lacp

                           LACP

   PORT   LACP      TRUNK     PORT      LACP      LACP
   NUMB   ENABLED   GROUP     STATUS    PARTNER   STATUS
   ----   -------   -------   -------   -------   -------
   19     Active    Trk1      Up        Yes       Success
   22     Active    Trk1      Up        Yes       Success

cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2+3 (2)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 200
Down Delay (ms): 0

802.3ad info
LACP rate: fast
Aggregator selection policy (ad_select): stable
Active Aggregator Info:
        Aggregator ID: 1
        Number of ports: 2
        Actor Key: 17
        Partner Key: 49
        Partner Mac Address: 84:34:97:49:74:c0

Slave Interface: eth0
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:90:5a:59:e0
Aggregator ID: 1
Slave queue ID: 0

Slave Interface: eth1
MII Status: up
Speed: 1000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 00:25:90:5a:59:e1
Aggregator ID: 1
Slave queue ID: 0

Are there any infos on switch side, why he only uses one link? If one link fails the failover to the second link is working fine without problems.

Any hints on that?

[Chrisd131313]

This post might help you understand why it is happening.. and it explains it better than I can :)

http://serverfault.com/questions/171333/how-exactly-specifically-does-layer-3-lacp-destination-address-hashing-work

I would take a look at the LACP options you have on RealServer to see if you can change the hashing, also you might be able to enable L4 load balancing on the switch...

trunk-load-balance l4-based

But this will only work is L4 is enabled on the other end of the link too, and if you switch supports it.

[Richard Brodie_1]

I believe the 2510G, as a plain level 2 switch only does Mac based SA/DA hashing. As you go higher up the range you can mix in the IP address and port number.

If your clients are coming through a router, so that the switch only sees a single hardware address, that could be a problem. I don't know of any easy way to debug this: you could take some captures and break out SA/DA Mac addresses, see if that's enlightening.

[norbertsch]

the clients are all connected to the same switch, so the source adress ist always not the same. so i think, the switch must hash to different channels/links, but he does...

and if i test with another machine, he is connected via uplink(1gig) to another switch, the result is the same. if i generate traffic via scp, the hole traffic(from all clients + scp server) is running over the same channel/link

[Chrisd131313]

Are you able to setup another VM on the destination server? if you connect some clients to this, do they all use the same LACP member then? It sounds like to me that the Destination MAC is whats causing the traffic to all go down the same link. get another destination MAC in the mix and see if this fixes it.

[norbertsch]

no i'm not able, its just a box for backups. if the problem is the destination mac, then the switch is only hashing with destination MAC and not with SA/DA MAC. sounds like a bug for me.

strange is... if copy data from the backup box to one client he is using the other link for outgoing traffic, but only if there is no incoming traffic from the same client.

[norbertsch · Accepted solution]

replaced hp switch with Cisco SG300-52, now it works