procurve LACP-trunk (4 ports) - iperf receives from 3 different nodes only on one port?

[Schwager]

Hallo together,

we setup a LACP trunk (4x 1GBit links) inside a procuve 4202vl connected to a linux system (bond mode 4, 802.3ad, hash-policy: layer2). The trunk itself works fine. 

Sending from the linux-system to 3 different hosts, i can see more than 2 GBIt/sec - more than 1 port is used for sending. This is fine.

If i setup the linux box as a iperf-server receiving from 3 other nodes at the same time, the 

commulated received througput is always exactly 1 GBit/sec(!). The linux box receives only on one port.

The manual ftp://ftp.hp.com/pub/networking/software/6400-5300-4200-3400-MgmtCfg-Oct2006-59906050.pdf

page 12-38, told me, the (procurve switch) output port  depends on a special calcuation (source/destination mac, amount of ports/trunk).

I have a trunk with 4(!) ports and 3 different hosts sending data to my linux box, but only (in total) 1GBit/sec incoming data is measured by iperf. Why? Is this bad luck my procurve "calculates" always the same outgoing (procurve) port to my linux box?

Is there a way to track down this calculation? Maybe with a nice procuve CLI command? Or do we have to buy a J9173A HP PCM+ V3 (as adviced me in the HP manual) to find a solution?

Best regards

Danny

BTW: The mac-adresses of the 3 sender nodes:

  LinuxNode: bond0: 00:07:e9:24:9d:fc

  Sender1: 00:25:90:e3:71:aa,

  Sender2: 00:25:90:e3:71:be

  Sender3: 00:50:56:A7:00:1E

[manuel.bitzi]

hi schwager

please post your lacp config and stats from the procurve.

But hopefully you know that this could be a normal behaviour? (probability: 1:64)

br

Manuel

[Schwager]

Hi Manuel,

here we are

I removed all entries except of Trk6 from the tables:

****** switch06# show trunks

 Load Balancing

  Port | Name                             Type      | Group Type 
  ---- + -------------------------------- --------- + ----- -----
  10   |                                  100/1000T | Trk6  LACP 
  11   |                                  100/1000T | Trk6  LACP 
  A10  |                                  100/1000T | Trk6  LACP 
  A11  |                                  100/1000T | Trk6  LACP 

******   switch06# show lacp

                           LACP

   PORT   LACP      TRUNK     PORT      LACP      LACP
   NUMB   ENABLED   GROUP     STATUS    PARTNER   STATUS
   ----   -------   -------   -------   -------   -------
   10     Active    Trk6      Up        Yes       Success
   11     Active    Trk6      Up        Yes       Success
   A10    Active    Trk6      Up        Yes       Success
   A11    Active    Trk6      Up        Yes       Success
   
   
****** switch06# show spanning-tree

    Multiple Spanning Tree (MST) Information

  STP Enabled   : Yes
  Force Version : MSTP-operation
  IST Mapped VLANs : 1-4094
  Switch MAC Address : 0016b9-72a100
  Switch Priority    : 32768
  Max Age  : 20
  Max Hops : 20
  Forward Delay : 15

  Topology Change Count  : 87          
  Time Since Last Change : 16 hours    

  CST Root MAC Address : 0016b9-72a100
  CST Root Priority    : 32768       
  CST Root Path Cost   : 0           
  CST Root Port        : This switch is root

  IST Regional Root MAC Address : 0016b9-72a100
  IST Regional Root Priority    : 32768       
  IST Regional Root Path Cost   : 0           
  IST Remaining Hops            : 20          

  Root Guard Ports : 
  TCN Guard Ports  : 
  Protected Ports :                                                          
  Filtered Ports :                                                          

                  |           Prio             | Designated    Hello         
  Port  Type      | Cost      rity  State      | Bridge        Time  PtP Edge
  ----- --------- + --------- ----- ---------- + ------------- ----- --- ----
  Trk6            | 20000     64    Forwarding | 0016b9-72a100 2     Yes Yes 

  

And some infos to my linux box:

************* Linux box
maintain:~# cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v3.2.5 (March 21, 2008)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 2000
Down Delay (ms): 200

802.3ad info
LACP rate: fast
Active Aggregator Info:
        Aggregator ID: 3
        Number of ports: 4
        Actor Key: 17
        Partner Key: 214
        Partner Mac Address: 00:16:b9:72:a1:00

Slave Interface: eth2
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:07:e9:24:9d:fc
Aggregator ID: 3

Slave Interface: eth3
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:07:e9:24:9d:fd
Aggregator ID: 3

Slave Interface: eth4
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:04:23:b7:0b:65
Aggregator ID: 3

Slave Interface: eth5
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:04:23:b7:0b:64
Aggregator ID: 3
  
  
maintain:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth5: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
    link/ether 00:07:e9:24:9d:fc brd ff:ff:ff:ff:ff:ff
3: eth4: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
    link/ether 00:07:e9:24:9d:fc brd ff:ff:ff:ff:ff:ff
4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:04:23:b7:07:d0 brd ff:ff:ff:ff:ff:ff
    inet 192.168.101.15/24 brd 192.168.101.255 scope global eth0
    inet6 fe80::204:23ff:feb7:7d0/64 scope link 
       valid_lft forever preferred_lft forever
5: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:04:23:b7:07:d1 brd ff:ff:ff:ff:ff:ff
    inet 192.168.91.15/24 brd 192.168.91.255 scope global eth1
    inet6 fe80::204:23ff:feb7:7d1/64 scope link 
       valid_lft forever preferred_lft forever
6: eth2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
    link/ether 00:07:e9:24:9d:fc brd ff:ff:ff:ff:ff:ff
7: eth3: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master bond0 state UP qlen 1000
    link/ether 00:07:e9:24:9d:fc brd ff:ff:ff:ff:ff:ff
23: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP 
    link/ether 00:07:e9:24:9d:fc brd ff:ff:ff:ff:ff:ff
    inet 192.168.135.22/24 brd 192.168.135.255 scope global bond0
    inet6 fe80::207:e9ff:fe24:9dfc/64 scope link 
       valid_lft forever preferred_lft forever

  

And some iperf stuff:

** 3 concurrent senders:

maintain:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 1.00 MByte (default)
------------------------------------------------------------
[  4] local 192.168.135.22 port 5001 connected with 192.168.135.11 port 48855
[  5] local 192.168.135.22 port 5001 connected with 192.168.135.12 port 60172
[  6] local 192.168.135.22 port 5001 connected with 192.168.135.99 port 58603
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec    444 MBytes    371 Mbits/sec
[  5]  0.0-10.0 sec    397 MBytes    332 Mbits/sec
[  6]  0.0-10.0 sec    420 MBytes    353 Mbits/sec

** 1 concurrent sender
maintain:~# iperf -s
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size: 1.00 MByte (default)
------------------------------------------------------------
[  4] local 192.168.135.22 port 5001 connected with 192.168.135.11 port 48856
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.0 sec  1.09 GBytes    936 Mbits/sec

best regards

Danny

[Schwager]

> But hopefully you know that this could be a normal behaviour? (probability: 1:64)

yes - but probability 1:64 ? How did cou calc this "1:64"? Is it not less?

regards

Danny

[manuel.bitzi]

hi schwager

your configuration and stats look all fine. Looks like you got into this 1:64 :-)

4 possible ports ^ 3 clients = 4^3 = 4*4*4 = 64

the probability is 1:64 that all clients hit the same lacp-port

br

Manuel

[Richard Brodie_1]

1 in 64 of all being on a particular port, 1 in 16 of all being on the same port.

These are genuinely different sending machines and not just multiple IPs bound to the same interface? If not and you really are that unlucky, you can play around with your MAC address, see what happens.*

*Don't blame me if you break your ARP cache, or your network administrator hates you.

[Schwager]

Maybe it has something to do with our network setup ? I don't think, but ..... or ?

Please check this ...

http://www.asciiflow.com/#Draw3430815847142859244/1592257237

regards

Danny