BAGG Group issue

[declan2777]

Hi there,

I am having an issue regardingBAGG group set up. There are 2 interfaces assigned to this BAGG group and they are not set as Active/Active. Ports are up but Line protocal is down on 1 side. I am wondering if there is any config missing from the set up. Please see below;

interface Bridge-Aggregation104
description *** NetscalerSDX 1/A ***
port link-type trunk
undo port trunk permit vlan 1
port trunk permit vlan 200 to 201 206 250 to 251 256 300 to 301 306 350 to 351 356 553 to 554
link-aggregation mode dynamic

interface Ten-GigabitEthernet1/1/4
port link-mode bridge
description *** NetscalerSDX 1 ***
port link-type trunk
undo port trunk permit vlan 1
port trunk permit vlan 200 to 201 206 250 to 251 256 300 to 301 306 350 to 351 356 553 to 554
port link-aggregation group 104

[Ivan_B]

Hi @declan2777 !

What is the second port in the BAGG104? run following commands and post their output here:

display link-aggregation verbose interface bri104
display interface bri104
display interface ten1/1/4
display interface <second port of BAGG104>
display current int <second port of BAGG104>

[declan2777]

Hi there,

Please see below;

<SW-JR-INT-X13-1-4>display link-aggregation verbose Bridge-Aggregation 104
Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing
Port Status: S -- Selected, U -- Unselected, I -- Individual
Port: A -- Auto port, M -- Management port, R -- Reference port
Flags: A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,
D -- Synchronization, E -- Collecting, F -- Distributing,
G -- Defaulted, H -- Expired

Aggregate Interface: Bridge-Aggregation104
Creation Mode: Manual
Aggregation Mode: Dynamic
Loadsharing Type: Shar
Management VLANs: None
System ID: 0x8000, 4cae-a3b4-8add
Local:
Port Status Priority Index Oper-Key Flag
XGE1/1/4(R) S 32768 5 5 {ACDEFG}
XGE2/1/4 U 32768 14 5 {ACG}
Remote:
Actor Priority Index Oper-Key SystemID Flag
XGE1/1/4 32768 0 0 0x8000, 0000-0000-0000 {DEF}
XGE2/1/4 32768 0 0 0x8000, 0000-0000-0000 {DEF}

<SW-JR-INT-X13-1-4>display interface bri104
Bridge-Aggregation104
Current state: UP
Line protocol state: UP
IP packet frame type: Ethernet II, hardware address: 4cae-a3b4-8b12
Description: *** NetscalerSDX 1/A ***
Bandwidth: 10000000 kbps
10Gbps-speed mode, full-duplex mode
Link speed type is autonegotiation, link duplex type is autonegotiation
PVID: 1
Port link-type: Trunk
VLAN Passing: 200-201, 206, 250-251, 256, 300-301, 306, 553-554
VLAN permitted: 200-201, 206, 250-251, 256, 300-301, 306, 350-351, 356, 553-554
Trunk port encapsulation: IEEE 802.1q
Last clearing of counters: Never
Last 300 seconds input: 0 packets/sec 0 bytes/sec 0%
Last 300 seconds output: 1 packets/sec 136 bytes/sec 0%
Input (total): 177201 packets, 15123574 bytes
20360 unicasts, 110643 broadcasts, 46198 multicasts, 0 pauses
Input (normal): 177201 packets, - bytes
20360 unicasts, 110643 broadcasts, 46198 multicasts, 0 pauses
Input: 0 input errors, 0 runts, 0 giants, 0 throttles
0 CRC, 0 frame, - overruns, 0 aborts
- ignored, - parity errors
Output (total): 4099602 packets, 474459985 bytes
25327 unicasts, 1936200 broadcasts, 2138075 multicasts, 0 pauses
Output (normal): 4099602 packets, - bytes
25327 unicasts, 1936200 broadcasts, 2138075 multicasts, 0 pauses
Output: 0 output errors, - underruns, 0 buffer failures
0 aborts, 0 deferred, 0 collisions, 0 late collisions
0 lost carrier, - no carrier

<SW-JR-INT-X13-1-4>display interface ten1/1/4
Ten-GigabitEthernet1/1/4
Current state: UP
Line protocol state: UP
IP packet frame type: Ethernet II, hardware address: 4cae-a3b4-8b12
Description: Link to NS-JRSDXROS01 (Rack U-13) Int 10/1
Bandwidth: 10000000 kbps
Loopback is not set
Media type is optical fiber, port is 10G_BASE_SR_SFP
Ethernet port mode: LAN
10Gbps-speed mode, full-duplex mode
Link speed type is autonegotiation, link duplex type is autonegotiation
Flow-control is not enabled
Maximum frame length: 9416
Allow jumbo frames to pass
Broadcast max-ratio: 100%
Multicast max-ratio: 100%
Unicast max-ratio: 100%
PVID: 1
MDI type: Automdix
Port link-type: Trunk
VLAN Passing: 200-201, 206, 250-251, 256, 300-301, 306, 553-554
VLAN permitted: 200-201, 206, 250-251, 256, 300-301, 306, 350-351, 356, 553-554
Trunk port encapsulation: IEEE 802.1q
Port priority: 0
Last link flapping: 1 hours 12 minutes 40 seconds
Last clearing of counters: Never
Current system time:2020-10-29 14:08:15 EU+01:00:00
Last time when physical state changed to up:2020-10-29 12:55:35 EU+01:00:00
Last time when physical state changed to down:2020-10-29 12:55:19 EU+01:00:00
Peak input rate: 199 bytes/sec, at 2020-10-21 12:26:28
Peak output rate: 759 bytes/sec, at 2020-10-22 15:27:32
Last 300 seconds input: 0 packets/sec 0 bytes/sec 0%
Last 300 seconds output: 1 packets/sec 120 bytes/sec 0%
Input (total): 95298 packets, 8395236 bytes
20099 unicasts, 52104 broadcasts, 23095 multicasts, 0 pauses
Input (normal): 95298 packets, - bytes
20099 unicasts, 52104 broadcasts, 23095 multicasts, 0 pauses
Input: 0 input errors, 0 runts, 0 giants, 0 throttles
0 CRC, 0 frame, - overruns, 0 aborts
- ignored, - parity errors
Output (total): 2889414 packets, 357016992 bytes
24443 unicasts, 889093 broadcasts, 1975878 multicasts, 0 pauses
Output (normal): 2889414 packets, - bytes
24443 unicasts, 889093 broadcasts, 1975878 multicasts, 0 pauses
Output: 0 output errors, - underruns, 0 buffer failures
0 aborts, 0 deferred, 0 collisions, 0 late collisions
0 lost carrier, - no carrier
IPv4 traffic statistics:
Last 0 seconds input rate: 0 packets/sec, 0 bytes/sec
Last 0 seconds output rate: 0 packets/sec, 0 bytes/sec
Input: 0 packets, 0 bytes
Output: 0 packets, 0 bytes
IPv6 traffic statistics:
Last 0 seconds input rate: 0 packets/sec, 0 bytes/sec
Last 0 seconds output rate: 0 packets/sec, 0 bytes/sec
Input: 0 packets, 0 bytes
Output: 0 packets, 0 bytes

<SW-JR-INT-X13-1-4>display interface ten2/1/4
Ten-GigabitEthernet2/1/4
Current state: UP
Line protocol state: DOWN(LAGG)
IP packet frame type: Ethernet II, hardware address: 4cae-a3b4-8356
Description: Link to NS-JRSDXROS01 (Rack U-13) Int 10/2
Bandwidth: 10000000 kbps
Loopback is not set
Media type is optical fiber, port is 10G_BASE_SR_SFP
Ethernet port mode: LAN
10Gbps-speed mode, full-duplex mode
Link speed type is autonegotiation, link duplex type is autonegotiation
Flow-control is not enabled
Maximum frame length: 9416
Allow jumbo frames to pass
Broadcast max-ratio: 100%
Multicast max-ratio: 100%
Unicast max-ratio: 100%
PVID: 1
MDI type: Automdix
Port link-type: Trunk
VLAN Passing: 200-201, 206, 250-251, 256, 300-301, 306, 553-554
VLAN permitted: 200-201, 206, 250-251, 256, 300-301, 306, 350-351, 356, 553-554
Trunk port encapsulation: IEEE 802.1q
Port priority: 0
Last link flapping: 1 hours 16 minutes 58 seconds
Last clearing of counters: Never
Current system time:2020-10-29 14:09:08 EU+01:00:00
Last time when physical state changed to up:2020-10-29 12:52:10 EU+01:00:00
Last time when physical state changed to down:2020-10-29 12:51:57 EU+01:00:00
Peak input rate: 195 bytes/sec, at 2020-10-21 12:25:11
Peak output rate: 246 bytes/sec, at 2020-10-21 23:11:57
Last 300 seconds input: 0 packets/sec 0 bytes/sec 0%
Last 300 seconds output: 0 packets/sec 17 bytes/sec 0%
Input (total): 81903 packets, 6728338 bytes
261 unicasts, 58539 broadcasts, 23103 multicasts, 0 pauses
Input (normal): 81903 packets, - bytes
261 unicasts, 58539 broadcasts, 23103 multicasts, 0 pauses
Input: 0 input errors, 0 runts, 0 giants, 0 throttles
0 CRC, 0 frame, - overruns, 0 aborts
- ignored, - parity errors
Output (total): 1210690 packets, 117501172 bytes
965 unicasts, 1047291 broadcasts, 162434 multicasts, 0 pauses
Output (normal): 1210690 packets, - bytes
965 unicasts, 1047291 broadcasts, 162434 multicasts, 0 pauses
Output: 0 output errors, - underruns, 0 buffer failures
0 aborts, 0 deferred, 0 collisions, 0 late collisions
0 lost carrier, - no carrier
IPv4 traffic statistics:
Last 0 seconds input rate: 0 packets/sec, 0 bytes/sec
Last 0 seconds output rate: 0 packets/sec, 0 bytes/sec
Input: 0 packets, 0 bytes
Output: 0 packets, 0 bytes
IPv6 traffic statistics:
Last 0 seconds input rate: 0 packets/sec, 0 bytes/sec
Last 0 seconds output rate: 0 packets/sec, 0 bytes/sec
Input: 0 packets, 0 bytes
Output: 0 packets, 0 bytes

interface Ten-GigabitEthernet2/1/4
port link-mode bridge
description Link to NS-JRSDXROS01 (Rack U-13) Int 10/2
port link-type trunk
undo port trunk permit vlan 1
port trunk permit vlan 200 to 201 206 250 to 251 256 300 to 301 306 350 to 351 356 553 to 554
port link-aggregation group 104
#

[Ivan_B]

Remote device doesn't send LACP PDUs:

Port Status Priority Index Oper-Key Flag
XGE1/1/4(R) S 32768 5 5 {ACDEFG}
XGE2/1/4 U 32768 14 5 {ACG}
Remote:
Actor Priority Index Oper-Key SystemID Flag
XGE1/1/4 32768 0 0 0x8000, 0000-0000-0000 {DEF}
XGE2/1/4 32768 0 0 0x8000, 0000-0000-0000 {DEF}

When you fix Netscaler's configuration, Instead of all zeroes (highlighted in red) you will see actual remote device's SystemID and LACP will bring up the aggregation on all ports. For now one of aggregation's port just works in standalone mode, that's why it's up. 

Hope this helps!

[declan2777]

Hi there,

So do you think the issue is with the SDX then? I removed the "link mode dynamic" command and SDX looked okay (remote device's SystemID) All ports & line protocol are UP.

Why by removing the dynamic did that work? Does Dynamic set the group as Active/Passive?

[Ivan_B · Accepted solution]

No, "link-aggregation mode dynamic" just disables LACP protocol and bundles all members in a BAGG unconditionally. No matter if settings on another side are correct and compatible, it just doesn't care. In some documents you can find this configuration as "no protocol" aggregation, we call it "static aggregation". It's not a good idea in general, because it's prone to errors. LACP not only negotiates an aggregation, it protects you from very dangerous misconfigurations that can possibly lead to Layer 2 loops. I recommend you to use LACP whenever possible. If that Netscaler device (according to BAGG's description string I call it "Netscaler", sorry if I am wrong here) uses LACP for aggregation, it's not configured properly. By using static aggregation you don't resolve the problem, you just make it worse, especially if those ports on Netscaler are not bundled in an aggregation or misconfigured.

Summarizing:

- If adjacent device (Netscaler or whatever) uses LACP, then it's misconfigured and you need to configure that device, not your switch.
- If adjacent device uses static aggregation (no protocol, without LACP, just static bundle) then you cannot use "link-aggregation mode dynamic" and must use also static aggregation on switch side (without "link-aggregation mode dynamic").
- In both cases you need to know your network and be 100% sure what you are doing.

P.S. As for "I removed the "link mode dynamic" command and SDX looked okay (remote device's SystemID)" You cannot see any SystemID if you disable LACP protocol, I am afraid you make a mistake here. Here is how a static aggregation look like:

Aggregate Interface: Bridge-Aggregation1
Aggregation Mode: Static
Loadsharing Type: Shar
Management VLANs: None

Port Status Priority Oper-Key
XGE1/2/0/26 U 32768 12 
XGE2/2/0/26 U 32768 12

As you can see there is no Remote part, no SystemID, simply because when you don't have any protocol to negotiate the aggregated channel, there is NOTHING that can report you SystemID of the adjacent device. We have no protocol, remember? No protocol - no information exchange. Pretty simple concept.

[declan2777]

Many thanks for your detailed response. Everything you said made 100% sense.!!

I did make a mistake regarding "P.S. As for "I removed the "link mode dynamic" command and SDX looked okay (remote device's SystemID)" You cannot see any SystemID if you disable LACP protocol, I am afraid you make a mistake here. Here is how a static aggregation look like:". I was looking at another device (my mistake)

Ill revert back to Netscaler and make sure config is 100%. Ivan thanks for all your help!!