LACP trunk to PaloAlto FW

[Michael135]

Hi I need some help, finding a LACP-error.

I have made some LACP trunks, all works fine, but one does not ...

hawe do I find some error-log ?

I have made a LACP :

interface Bridge-Aggregation20
port link-type trunk
undo port trunk permit vlan 1
port trunk permit vlan 3 to 4
link-aggregation mode dynamic

interface Ten-GigabitEthernet1/1/5
port link-mode bridge
port link-type trunk
undo port trunk permit vlan 1
port trunk permit vlan 3 to 4
port link-aggregation group 20

interface Ten-GigabitEthernet2/1/5
port link-mode bridge
port link-type trunk
undo port trunk permit vlan 1
port trunk permit vlan 3 to 4
port link-aggregation group 20

but, a simple ping-test have about 60% packet-loss :-(

If I disable one interface, it works fine..

any ideers ??

[parnassus]

What is the output of the display link-aggregation verbose Bridge-Aggregation 20 command?

[Michael135]

[dc2.core1]display link-aggregation verbose Bridge-Aggregation 20
Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing
Port Status: S -- Selected, U -- Unselected,
I -- Individual, * -- Management port
Flags: A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,
D -- Synchronization, E -- Collecting, F -- Distributing,
G -- Defaulted, H -- Expired

Aggregate Interface: Bridge-Aggregation20
Aggregation Mode: Dynamic
Loadsharing Type: Shar
Management VLAN : None
System ID: 0x8000, e8f7-242c-9537
Local:
Port Status Priority Oper-Key Flag
--------------------------------------------------------------------------------
XGE1/1/5 S 32768 1 {ACDEF}
XGE2/1/5 S 32768 1 {ACDEF}
Remote:
Actor Partner Priority Oper-Key SystemID Flag
--------------------------------------------------------------------------------
XGE1/1/5 32 32768 48 0x8000, d4f4-be76-9401 {CDEF}
XGE2/1/5 33 32768 48 0x8000, d4f4-be76-9401 {CDEF}

[parnassus]

Interesting.

Note how the Remote partner (the Palo Alto Firewall with MAC Addressd 4f:4:be:76:94:01) is shown as {CDEF} flagged and not as {ACDEF} flagged as it should be (the A letter means "LACP Activity" as per command provided legend).

Try to permit VLAN 1 (the Management VLAN) instead of setting it as not permitted either on interface Bridge-Aggregation20 and on its members (port trunk permit vlan 1).

Eventually re-do the BAGG configuration from scratch (remove cables, reconfigure, re-connect cables) since in Comware based Switches the order of steps for Port Trunking configuration is important (to avoid mismatches between the Logical Interface BAGG20 and its members physical ports).

Also provide configuration of LACP Port Trunking on the Palo Alto Firewall side <-- that could be the very culprit.

Do these commands to start troubleshooting (Switch side):

• display interface brief | include UP (limiting to copy and paste the relevant physical interfaces XGE1/1/5 and XGE2/1/5 and the logical interface BAGG20).
• display lldp neighbor-information list
• display link-aggregation summary
• reset lacp statistics
• display link-aggregation summary (again to see how zeroed statistics change, if any).

Which Switch are you using (Model, Firmware version)?

What is the Palo Alto Firewall configuration regarding its two ports (LACP) Port Trunking=

[Michael135]

hi

yes, I have tryed to start over (removed all, and start over...)

but the result is the same (allmost), my "ping-test" have a bit too high packet-loss :

80 packets transmitted, 40 received, 50% packet loss, time 79044ms
rtt min/avg/max/mdev = 0.536/0.605/0.775/0.056 ms

but I did change a setting in the PaloAlto, witch gave this : (picture uploadet)

display link-aggregation verbose Bridge-Aggregation 20
Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing
Port Status: S -- Selected, U -- Unselected,
I -- Individual, * -- Management port
Flags: A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation,
D -- Synchronization, E -- Collecting, F -- Distributing,
G -- Defaulted, H -- Expired

Aggregate Interface: Bridge-Aggregation20
Aggregation Mode: Dynamic
Loadsharing Type: Shar
Management VLAN : None
System ID: 0x8000, e8f7-242c-9537
Local:
Port Status Priority Oper-Key Flag
--------------------------------------------------------------------------------
XGE1/1/5 S 32768 1 {ACDEF}
XGE2/1/5 S 32768 1 {ACDEF}
Remote:
Actor Partner Priority Oper-Key SystemID Flag
--------------------------------------------------------------------------------
XGE1/1/5 32 32768 48 0x8000, d4f4-be76-9401 {ACDEF}
XGE2/1/5 33 32768 48 0x8000, d4f4-be76-9401 {ACDEF}

display interface brief | include UP :

BAGG20 UP 20G(a) F(a) T 1 Uplink til dc2.fw1
XGE1/1/5 UP 10G(a) F(a) T 1
XGE2/1/5 UP 10G(a) F(a) T 1

display lldp neighbor-information list
Chassis ID : * -- -- Nearest nontpmr bridge neighbor
# -- -- Nearest customer bridge neighbor
Default -- -- Nearest bridge neighbor
System Name Local Interface Chassis ID Port ID
dc2.man XGE1/1/23 e8f7-2446-2282 Ten-GigabitEthernet1/0/49
Multi 7.0 XGE1/1/24 0017-a4b6-c200 200
dc2.core1 XGE1/2/23 e8f7-242c-9537 Ten-GigabitEthernet2/2/24
dc2.core1 XGE1/2/24 e8f7-242c-9537 Ten-GigabitEthernet2/2/23
dc2.man M-GE0/0/0 e8f7-2446-2282 GigabitEthernet1/0/2
dc2.man XGE2/1/23 e8f7-2446-2282 Ten-GigabitEthernet1/0/50
dc2.core1 XGE2/2/23 e8f7-242c-9537 Ten-GigabitEthernet1/2/24
dc2.core1 XGE2/2/24 e8f7-242c-9537 Ten-GigabitEthernet1/2/23
[dc2.core1]display link-aggregation summary
Aggregation Interface Type:
BAGG -- Bridge-Aggregation, BLAGG -- Blade-Aggregation, RAGG -- Route-Aggregation
Aggregation Mode: S -- Static, D -- Dynamic
Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing
Actor System ID: 0x8000, e8f7-242c-9537

AGG AGG Partner ID Selected Unselected Individual Share
Interface Mode Ports Ports Ports Type
--------------------------------------------------------------------------------
BAGG1 D 0x8000, 0000-0000-0000 0 2 0 Shar
BAGG5 D 0x8000, e8f7-2446-2282 2 0 0 Shar
BAGG20 D 0x8000, d4f4-be76-9401 2 0 0 Shar
BAGG21 D 0x8000, 0000-0000-0000 0 2 0 Shar

it is running on at HPE FF 5930-4Slot Switch, Software Version 7.1.045, Release 2422P01

the normal setup for a PaloAlto : https://live.paloaltonetworks.com/t5/Configuration-Articles/How-to-Configure-LACP/ta-p/65837 

[parnassus]

OK, better...now the run show lacp aggregate-ethernet all on the Palo Alto Firewall what output produces?

[Michael135]

fw1(active)> show lacp aggregate-ethernet all

LACP:

**********************************************************************************
AE group: ae1
Members: Bndl Rx state Mux state Sel state
ethernet1/17 yes Current Tx_Rx Selected
ethernet1/18 yes Current Tx_Rx Selected
Status: Enabled
Mode: Active
Rate: Slow
Max-port: 8
Fast-failover: Disabled
Pre-negotiation: Disabled
Local: System Priority: 32768
System MAC: d4:f4:be:76:94:01
Key: 48
Partner: System Priority: 32768
System MAC: e8:f7:24:2c:95:37
Key: 1
Port State
--------------------------------------------------------------------------------
Interface Port
Number Priority Mode Rate Key State
--------------------------------------------------------------------------------
ethernet1/17 32 32768 Active Slow 48 0x3D
Partner 5 32768 Active Slow 1 0x3D

ethernet1/18 33 32768 Active Slow 48 0x3D
Partner 210 32768 Active Slow 1 0x3D

Port Counters
--------------------------------------------------------------------------------
Interface LACPDUs Marker Marker Response Error
Sent Recv Sent Recv Sent Recv Unknown Illegal
--------------------------------------------------------------------------------
ethernet1/17 531 491 0 0 0 0 0 0
ethernet1/18 529 489 0 0 0 0 0 0

[parnassus]

Where exactly are you executing the Ping command (source of the ping: a VM on the ESXi, the ESXi itself, the 5930 Switch itself or what else)? and what's about the destination of the ping command?

Are you working on a IRF Stack of 5930 switches or the 5930 is just a single unit?

[Michael135]

the ping-test is all running from our central FW (two switches "away", from this setup). I have two destinations (in this setup) boath configured as a lacp trunks direct connectet to this 5930-IRF stack... one set (other produckt) running just fine and the other (this PA firewalls) a bit of packet-loss...

So all testing are from and to real network aplaince-boxes (no vm)

[parnassus]

The output of the command show lacp aggregate-ethernet all on the PA looks good (the exchange of LACPDUs between both Trunk's ends happens and also the "ae1" LACP Trunk is shown as Active and as Enabled...against the HPE switch, as the Partner MAC Address shows).

What's about checking "ae1" Aggregate Ethernet Interface's Link Settings (Link Speed, Link Duplex and Link State) for any possible mismatch?

What's about performing the ICMP Ping from the HPE switch against the PA Firewall? do you obtain a similar or identical pattern (50% packet loss) or what?

On both Switch and Firewall sides: do all involved physical interfaces report link flapping on transmission errors?

Sorry, what do you mean with "two switches "away", from this setup"? Isn't the HPE 5930 directly trunked to the PA Firewall (so why you pointed out "two switches away"?) as we expected?

What I understood is that the PA Firewall is at one LACP Trunk's end and another Switch is at another LACP Trunk's end, The HPE 5930 is just sitting at the center...separately providing these two independent LACP Trunks. Is that right?