LACP randomly fail between hpe5700 and cisco 6509

lozair31 · ‎12-15-2016

Hi,

we encounter here a LACP problem between two devices, this LACP is constructed with two Ten Gigabit interfaces on each side. :

HPE5700-32XGT-8XG-2QSFP+ (interfacename =Bridgeaggregation33 [ XGE1/0/33 + XGE2/0/33 ])
Cisco 6509 (interfacename=Po14 [ Te1/3 + Te3/3 ]).

Randomly, once a week, the Cisco 6509 set PortChannel 14 in errdisabled state and cut the link to the HPE5700 with the following message :

CET: %PM-SPSTBY-4-ERR_DISABLE: channel-misconfig error detected on Te1/3, putting Te1/3 in err-disable state
CET: %PM-SPSTBY-4-ERR_DISABLE: channel-misconfig error detected on Te3/3, putting Te3/3 in err-disable state
%PM-SPSTBY-4-ERR_DISABLE: channel-misconfig error detected on Po14, putting Te1/3 in err-disable state

On the same time these logs appears on HPE5700 :

<190>Dec  3 02:33:59 2016 SWITCH %%10LAGG/6/LAGG_INACTIVE_CONFIGURATION: Member port XGE1/0/33 of aggregation group BAGG33 changed to the inactive state, because the aggregation configuration of the port is incorrect.
<190>Dec  3 02:34:00 2016 SWITCH %%10LAGG/6/LAGG_ACTIVE: Member port XGE1/0/33 of aggregation group BAGG33 changed to the active state.
<190>Dec  3 02:34:00 2016 SWITCH %%10LAGG/6/LAGG_INACTIVE_CONFIGURATION: Member port XGE2/0/33 of aggregation group BAGG33 changed to the inactive state, because the aggregation configuration of the port is incorrect.
<190>Dec  3 02:34:00 2016 SWITCH %%10LAGG/6/LAGG_INACTIVE_PHYSTATE: Member port XGE1/0/33 of aggregation group BAGG33 changed to the inactive state, because the physical state of the port is down.
<187>Dec  3 02:34:00 2016 SWITCH %%10IFNET/3/PHY_UPDOWN: Ten-GigabitEthernet2/0/33 link status is down.
<189>Dec  3 02:34:00 2016 SWITCH %%10IFNET/5/LINK_UPDOWN: Line protocol on the interface Ten-GigabitEthernet2/0/33 is down.
<187>Dec  3 02:34:00 2016 SWITCH %%10IFNET/3/PHY_UPDOWN: Ten-GigabitEthernet1/0/33 link status is down.
<189>Dec  3 02:34:00 2016 SWITCH %%10IFNET/5/LINK_UPDOWN: Line protocol on the interface Ten-GigabitEthernet1/0/33 is down.
<190>Dec  3 02:34:00 2016 SWITCH %%10LAGG/6/LAGG_INACTIVE_PHYSTATE: Member port XGE1/0/33 of aggregation group BAGG33 changed to the inactive state, because the physical state of the port is down.
<187>Dec  3 02:34:00 2016 SWITCH %%10IFNET/3/PHY_UPDOWN: Bridge-Aggregation33 link status is down.
<189>Dec  3 02:34:00 2016 SWITCH %%10IFNET/5/LINK_UPDOWN: Line protocol on the interface Bridge-Aggregation33 is down.

Reading the logs, it seems there is an event on HPE5700 around bridge aggregation 33 port members. This event seems produce the err-disable protection on the cisco side..After five minutes the Cisco leave the err-disable state on the portchannel 14 and all works like a charm...

.We have read the docs and guides about LACP but we can't identify the reason of this random event on the HPE5700. I want precise there is no errors on the links between cisco and HP.

We use the following configuratoin on the HP :

Config

interface Ten-GigabitEthernet1/0/33
description "XXXXXXXXXX"
port link-type trunk
port trunk permit vlan all
port link-aggregation group 33

interface Ten-GigabitEthernet2/0/33
description "XXXXXXXXXX"
port link-type trunk
port trunk permit vlan all
port link-aggregation group 33

interface Bridge-Aggregation33
description "YYYYYYYYY"
port link-type trunk
port trunk permit vlan all
link-aggregation mode dynamic

State

display interface Ten-GigabitEthernet 1/0/33
Ten-GigabitEthernet1/0/33
Current state: UP
Line protocol state: UP
IP packet frame type: Ethernet II, hardware address: e8f7-24b9-411c
Description: "palays port"
Bandwidth: 10000000 kbps
Loopback is not set
Media type is optical fiber, port hardware type is 10G_BASE_SR_SFP
10Gbps-speed mode, full-duplex mode
Link speed type is autonegotiation, link duplex type is autonegotiation
Flow-control is not enabled
Maximum frame length: 10000
Allow jumbo frames to pass
Broadcast max-ratio: 100%
Multicast max-ratio: 100%
Unicast max-ratio: 100%
PVID: 1
MDI type: Automdix
Port link-type: Trunk

display interface Ten-GigabitEthernet 2/0/33
Ten-GigabitEthernet2/0/33
Current state: UP
Line protocol state: UP
IP packet frame type: Ethernet II, hardware address: e8f7-24b6-9ae3
Description: "palays port"
Bandwidth: 10000000 kbps
Loopback is not set
Media type is optical fiber, port hardware type is 10G_BASE_SR_SFP
10Gbps-speed mode, full-duplex mode
Link speed type is autonegotiation, link duplex type is autonegotiation
Flow-control is not enabled
Maximum frame length: 10000
Allow jumbo frames to pass
Broadcast max-ratio: 100%
Multicast max-ratio: 100%
Unicast max-ratio: 100%
PVID: 1
MDI type: Automdix
Port link-type: Trunk

display interface Bridge-Aggregation 33
Bridge-Aggregation33
Current state: UP
IP packet frame type: Ethernet II, hardware address: e8f7-24b9-411c
Description: "Aggregation de liens vers palays"
Bandwidth: 20000000 kbps
20Gbps-speed mode, full-duplex mode
Link speed type is autonegotiation, link duplex type is autonegotiation
PVID: 1
Port link-type: Trunk

Thanks for your help or advices

peyrache · ‎12-15-2016

hello

shall you post firmware versions for both please

thanks

JY

Vince-Whirlwind · ‎12-15-2016

What if you use link aggregation-mode static instead of dynamic?

lozair31 · ‎12-16-2016

the firmware versions :

HPE5700 :

Current software images:
!   flash:/5700-cmw710-boot-r2422p01.bin
!   flash:/5700-cmw710-system-r2422p01.bin

Cisco 6509 :

Image: Software: s72033_rp-ADVENTERPRISEK9-M, 12.2(33)SXJ9, RELEASE SOFTWARE (fc4)
!Image: Compiled: Mon 16-Mar-15 14:07 by prod_rel_team
!Image: disk1:s72033-adventerprisek9-mz.122-33.SXJ9.bin

Hope this can help

lozair31 · ‎12-16-2016

We don't test it.

This link join network core router to our server switch stack.......

Do you think this can resolve the issue ?

parnassus · ‎12-16-2016

What is the display link-aggregation verbose command's output on the HPE 5700?

Mumble Mumble...you cited stack ("This link join network core router to our server switch stack......."), do you have an IRF Stack of HPE 5700 units or just a single HPE 5700 Switch?

I'm not an HPE Employee

peyrache · ‎12-16-2016

should be great to be in Static in both side if problem still occured

debug lacp on cisco side witl help to get more details

JY

lozair31 · ‎12-16-2016

display link-aggregation verbose Bridge-Aggregation 33
Loadsharing Type: Shar -- Loadsharing, NonS -- Non-Loadsharing 
Port Status: S -- Selected, U -- Unselected,  
             I -- Individual, * -- Management port 
Flags:  A -- LACP_Activity, B -- LACP_Timeout, C -- Aggregation, 
        D -- Synchronization, E -- Collecting, F -- Distributing, 
        G -- Defaulted, H -- Expired 

Aggregate Interface: Bridge-Aggregation33
Aggregation Mode: Dynamic
Loadsharing Type: Shar
Management VLAN : None
System ID: 0x8000, e8f7-24b9-40d3
Local: 
  Port             Status  Priority Oper-Key  Flag
--------------------------------------------------------------------------------
  XGE1/0/33        S       32768    1         {ACDEF}
  XGE2/0/33        S       32768    1         {ACDEF}
Remote: 
  Actor            Partner Priority Oper-Key  SystemID               Flag   
--------------------------------------------------------------------------------
  XGE1/0/33        260     32768    14        0x8000, 0011-5dbe-3000 {ACDEF}
  XGE2/0/33        772     32768    14        0x8000, 0011-5dbe-3000 {ACDEF}

Yes we have an IRF Stack of two HPE switch

display irf
MemberID    Role    Priority  CPU-Mac         Description
 *+1        Master  32        00e0-fc0f-8c02  ---
   2        Standby 1         00e0-fc0f-8c03  ---
--------------------------------------------------
 * indicates the device is the master.
 + indicates the device through which the user logs in.

 The bridge MAC of the IRF is: e8f7-24b9-40d3
 Auto upgrade                : yes
 Mac persistent              : always
 Domain ID                   : 0
 IRF mode                    : normal

lozair31 · ‎12-16-2016

We will attempt this config if the error come again

thks for your advice

VoIP-Buddy · ‎12-19-2016

The only gotcha with static link-agg is that the links never go down. If something happens, you'll be sending packets into a abyss.

I would debug lacp on the 5700's as well to see what is causing this event.

You might also want to check Spanning Tree, if it is enabled, to see if one of the ports is blocked.

David

I work for HPE in Aruba Technical Support

lozair31 · ‎12-19-2016

thks for the reply

i can't enable debug lacp on 5700...

no more information, is there specific command to get debug events sent to syslog like other events on hp5700 ?

same at cisco level i had enable debugging lacp events and error but no more results...

we think about set the cisco in LACP passive mode and kept hp5700 in LACP active mode but i don't know if that can have an effect on our problem..

VoIP-Buddy · ‎12-19-2016

Hi!

Use this command... > debug link-aggregation lacp packet You can then choose all, send, or receive.
If you are sending debug to syslog you should be all set. That is done in the information-center.

Regards,

David

I work for HPE in Aruba Technical Support

LostedMan · ‎12-28-2016

This may help, it may not. Attached is the pertinent config from our 3Com XRN switch stack doing EtherChannel with our Cisco 6509E VSS cluster. Comware version 3.03.02s168p23, Cisco IOS version 15.1(1)SY.

Cisco

interface Port-channel7
description \\ Link to Ground floor //
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
!
interface TenGigabitEthernet1/2/1
description \\ Ground Floor 3Com XRN //
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
logging event link-status
channel-protocol lacp
channel-group 7 mode active
!
interface TenGigabitEthernet2/2/1
description \\ Ground Floor 3Com XRN //
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
logging event link-status
channel-protocol lacp
channel-group 7 mode active

3Com

interface TenGigabitEthernet1/1/1
port link-type trunk
port trunk permit vlan all
undo jumboframe enable
gvrp
lacp enable
#
interface TenGigabitEthernet2/1/1
port link-type trunk
port trunk permit vlan all
undo jumboframe enable
gvrp
lacp enable

lozair31 · ‎01-03-2017

thanks for your reply...

it seems we are running the same config here.....

lozair31 · ‎01-03-2017

thanks for the information

We have enabled the debug en we get logs about lacp on our syslog server.

It seems the cisco break the link but i'm not sure about that

attached logs from H3C 5700 with lacp debug activated.

It seems the H3c stack receive lacp information (FSM.RX) from cisco saying " i disable the port" :

02:31:56 2016 sw-csn-srv %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.RX CURRENT-->PORT_DISABLED, Port_Disabled
02:31:57 2016 sw-csn-srv %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.RX CURRENT-->PORT_DISABLED, Port_Disabled

That seems indicate that it's the cisco wich cut the link.....

I can't enable lacp debug on 6509, the debug lacp command seems not exist on our Cisco OS

s72033_rp-ADVENTERPRISEK9-M, 12.2(33)SXJ9, RELEASE SOFTWARE (fc4)

peyrache · ‎01-03-2017

hi

this configuration did ever worked or always experimented this issue ?

what are cisco version ?

JY

peyrache · ‎01-03-2017

shall you post

show tech from cisco device please

JY

lozair31 · ‎01-03-2017

this configuration is up since 3 months now and we encounter this random problem immediately after we setup this

but this problem seems to be linked to event on the network.....

This problem occur randomly once a week.

we have this link down mostly at the same time range, approximatively 2:30 AM

lozair31 · ‎01-03-2017

------------------ show clock ------------------

15:12:10.526 CET Tue Jan 3 2017

------------------ show version ------------------

Cisco IOS Software, s72033_rp Software (s72033_rp-ADVENTERPRISEK9-M), Version 12.2(33)SXJ9, RELEASE SOFTWARE (fc4)
Technical Support: http://www.cisco.com/techsupport
Copyright (c) 1986-2015 by Cisco Systems, Inc.
Compiled Mon 16-Mar-15 14:07 by prod_rel_team

ROM: System Bootstrap, Version 12.2(17r)S2, RELEASE SOFTWARE (fc1)

XXXX uptime is 44 weeks, 6 days, 3 hours, 5 minutes
Uptime for this control processor is 44 weeks, 6 days, 3 hours, 2 minutes
Time since XXXX switched to active is 44 weeks, 6 days, 3 hours, 5 minutes
System returned to ROM by reload at 08:28:08 CET Wed Oct 5 2011 (SP by reload)
System restarted at 12:06:44 CET Wed Feb 24 2016
System image file is "disk1:s72033-adventerprisek9-mz.122-33.SXJ9.bin"
Last reload reason: bus error at PC 0x41687C74, address 0xD0D0E3D

This product contains cryptographic features and is subject to United
States and local country laws governing import, export, transfer and
use. Delivery of Cisco cryptographic products does not imply
third-party authority to import, export, distribute or use encryption.
Importers, exporters, distributors and users are responsible for
compliance with U.S. and local country laws. By using this product you
agree to comply with applicable laws and regulations. If you are unable
to comply with U.S. and local laws, return this product immediately.

A summary of U.S. laws governing Cisco cryptographic products may be found at:
http://www.cisco.com/wwl/export/crypto/tool/stqrg.html

If you require further assistance please contact us by sending email to
export@cisco.com.

cisco WS-C6509-E (R7000) processor (revision 1.1) with 458720K/65536K bytes of memory.
Processor board ID SAL08290QNN
SR71000 CPU at 600Mhz, Implementation 0x504, Rev 1.2, 512KB L2 Cache
Last reset from power-on
116 Virtual Ethernet interfaces
90 Gigabit Ethernet interfaces

Here the show version result with os version...

peyrache · ‎01-03-2017

should be hardware issue

shall you post

display diag from HP switch

show tech from Cisco switch

lozair31 · ‎01-04-2017

Hi

I can't publish all the conf of our cisco......

reading show tech or show diags there is no physical errors on the links.

We think about STP problem/event...

We successfully enable lacp and stp logs on cisco for now and we are wainting the next random error.

thks for your help and advices

regards

lozair31 · ‎01-19-2017

The error come again on our switchs.....

Reading and analyzing the logs it seems linked to server reboot.

The attached doc show our network topology.

The server reboot all the nights à 2:30 AM.

Every 3 weeks, this reboot is followed by a channel-misconfig error on Port-channel 14 of the Cisco 6500.

But the logs below seems indicate it's the hp5700 which cut the port for unknown reason and when the port come up in the followiong seconds there is a channel misconfiguration detection at the Cisco level.

Logs for the H3C 5700 switch :

Jan 19 02:31:50 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.RX CURRENT--
Jan 19 02:31:50 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/35 FSM.PTX PERIODIC_TX--
Jan 19 02:31:50 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/35 FSM.PTX SLOW_PERIODIC--
Jan 19 02:31:50 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/33.receive. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=0011-5dbe-3000, key=0xe, pri=0x8000, port-index=0x304, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x1, pri=0x8000, port-index=0xee, state=0x3d Collector: type=3, len=16, col-max-delay=0x8000 Terminator: type=0, len=0
Jan 19 02:31:50 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/35.send. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x2, pri=0x8000, port-index=0xf0, state=0x3d Partner: type=2, len=20, sys-pri=0x1, sys-mac=02a0-9895-db05, key=0x1, pri=0x0, port-index=0x2, state=0x3d Collector: type=3, len=16, col-max-delay=0x0 Terminator: type=0, len=0
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.ACTOR-CHURN ACTOR_CHURN_MONITOR--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.ACTOR-CHURN NO_ACTOR_CHURN--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.ACTOR-CHURN NO_ACTOR_CHURN--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.MUX ATTACHED--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.MUX ATTACHED--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.MUX ATTACHED--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.MUX COLLECTING_DISTRIBUTING--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.MUX COLLECTING_DISTRIBUTING--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.MUX DETACHED--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.MUX WAITING--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.PARTNER-CHURN NO_PARTNER_CHURN--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.RX CURRENT--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/33 FSM.RX CURRENT--
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/33.receive. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=0011-5dbe-3000, key=0xe, pri=0x8000, port-index=0x304, state=0x1 Partner: type=2, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x1, pri=0x8000, port-index=0xee, state=0x3d Collector: type=3, len=16, col-max-delay=0x8000 Terminator: type=0, len=0
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/33.send. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x1, pri=0x8000, port-index=0xee, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=0011-5dbe-3000, key=0xe, pri=0x8000, port-index=0x304, state=0x9 Collector: type=3, len=16, col-max-delay=0x0 Terminator: type=0, len=0
Jan 19 02:31:51 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/33.send. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x1, pri=0x8000, port-index=0xee, state=0x5 Partner: type=2, len=20, sys-pri=0x8000, sys-mac=0011-5dbe-3000, key=0xe, pri=0x8000, port-index=0x304, state=0x9 Collector: type=3, len=16, col-max-delay=0x0 Terminator: type=0, len=0
Jan 19 02:31:51 2017 hp5700 %%10LLDP/6/LLDP_DELETE_NEIGHBOR: -Slot=2; Nearest bridge agent neighbor deleted on Port Ten-GigabitEthernet2/0/33 (IfIndex 238), Chassis ID is 0011-5dbe-3000, Port ID is Te3/3.
Jan 19 02:31:52 2017 hp5700 %%10IFNET/3/PHY_UPDOWN: Bridge-Aggregation33 link status is down.
Jan 19 02:31:52 2017 hp5700 %%10IFNET/3/PHY_UPDOWN: Ten-GigabitEthernet1/0/33 link status is down.
Jan 19 02:31:52 2017 hp5700 %%10IFNET/3/PHY_UPDOWN: Ten-GigabitEthernet2/0/33 link status is down.
Jan 19 02:31:52 2017 hp5700 %%10IFNET/5/LINK_UPDOWN: Line protocol on the interface Bridge-Aggregation33 is down.
Jan 19 02:31:52 2017 hp5700 %%10IFNET/5/LINK_UPDOWN: Line protocol on the interface Ten-GigabitEthernet1/0/33 is down.
Jan 19 02:31:52 2017 hp5700 %%10IFNET/5/LINK_UPDOWN: Line protocol on the interface Ten-GigabitEthernet2/0/33 is down.
Jan 19 02:31:52 2017 hp5700 %%10IFNET/5/LINK_UPDOWN: Line protocol on the interface Ten-GigabitEthernet2/0/33 is down.
Jan 19 02:31:52 2017 hp5700 %%10IFNET/5/LINK_UPDOWN: Line protocol on the interface Ten-GigabitEthernet2/0/33 is up.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/6/LAGG_ACTIVE: Member port XGE1/0/33 of aggregation group BAGG33 changed to the active state.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/6/LAGG_ACTIVE: Member port XGE2/0/33 of aggregation group BAGG33 changed to the active state.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/6/LAGG_INACTIVE_CONFIGURATION: Member port XGE1/0/33 of aggregation group BAGG33 changed to the inactive state, because the aggregation configuration of the port is incorrect.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/6/LAGG_INACTIVE_CONFIGURATION: Member port XGE1/0/33 of aggregation group BAGG33 changed to the inactive state, because the aggregation configuration of the port is incorrect.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/6/LAGG_INACTIVE_CONFIGURATION: Member port XGE2/0/33 of aggregation group BAGG33 changed to the inactive state, because the aggregation configuration of the port is incorrect.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/6/LAGG_INACTIVE_PHYSTATE: Member port XGE2/0/33 of aggregation group BAGG33 changed to the inactive state, because the physical state of the port is down.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of member port up/down.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of member port up/down.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Event: Group 33 updated because of partner port other reason.
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.ACTOR-CHURN ACTOR_CHURN_MONITOR--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.ACTOR-CHURN NO_ACTOR_CHURN--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.ACTOR-CHURN NO_ACTOR_CHURN--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.MUX ATTACHED--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.MUX ATTACHED--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.MUX ATTACHED--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.MUX COLLECTING_DISTRIBUTING--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.MUX COLLECTING_DISTRIBUTING--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.MUX DETACHED--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.MUX WAITING--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.PARTNER-CHURN NO_PARTNER_CHURN--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.RX CURRENT--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.RX CURRENT--
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Packet: PACKET.Ten-GigabitEthernet1/0/33.receive. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=0011-5dbe-3000, key=0xe, pri=0x8000, port-index=0x104, state=0x1 Partner: type=2, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x1, pri=0x8000, port-index=0x21, state=0x3d Collector: type=3, len=16, col-max-delay=0x8000 Terminator: type=0, len=0
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Packet: PACKET.Ten-GigabitEthernet1/0/33.send. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x1, pri=0x8000, port-index=0x21, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=0011-5dbe-3000, key=0xe, pri=0x8000, port-index=0x104, state=0x9 Collector: type=3, len=16, col-max-delay=0x0 Terminator: type=0, len=0
Jan 19 02:31:52 2017 hp5700 %%10LAGG/7/Packet: PACKET.Ten-GigabitEthernet1/0/33.send. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x1, pri=0x8000, port-index=0x21, state=0x5 Partner: type=2, len=20, sys-pri=0x8000, sys-mac=0011-5dbe-3000, key=0xe, pri=0x8000, port-index=0x104, state=0x9 Collector: type=3, len=16, col-max-delay=0x0 Terminator: type=0, len=0
Jan 19 02:31:52 2017 hp5700 %%10LLDP/6/LLDP_DELETE_NEIGHBOR: Nearest bridge agent neighbor deleted on Port Ten-GigabitEthernet1/0/33 (IfIndex 33), Chassis ID is 0011-5dbe-3000, Port ID is Te1/3.
Jan 19 02:31:54 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/34 FSM.RX CURRENT--
Jan 19 02:31:54 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/34.receive. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=4431-92d6-82de, key=0x11, pri=0x8000, port-index=0x36, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x6, pri=0x8000, port-index=0xef, state=0x3d Collector: type=3, len=16, col-max-delay=0x0 Terminator: type=0, len=0
Jan 19 02:31:56 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/35 FSM.RX CURRENT--
Jan 19 02:31:56 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/35.receive. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x1, sys-mac=02a0-9895-db05, key=0x1, pri=0x0, port-index=0x2, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x2, pri=0x8000, port-index=0xf0, state=0x3d Collector: type=3, len=16, col-max-delay=0x64 Terminator: type=0, len=0
Jan 19 02:31:57 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.PTX PERIODIC_TX--
Jan 19 02:31:57 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/33 FSM.PTX SLOW_PERIODIC--
Jan 19 02:31:57 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/35 FSM.RX CURRENT--
Jan 19 02:31:57 2017 hp5700 %%10LAGG/7/Packet: PACKET.Ten-GigabitEthernet1/0/35.receive. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x1, sys-mac=02a0-9895-db05, key=0x1, pri=0x0, port-index=0x1, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x2, pri=0x8000, port-index=0x23, state=0x3d Collector: type=3, len=16, col-max-delay=0x64 Terminator: type=0, len=0
Jan 19 02:31:58 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/34 FSM.PTX PERIODIC_TX--
Jan 19 02:31:58 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/34 FSM.PTX SLOW_PERIODIC--
Jan 19 02:31:58 2017 hp5700 %%10LAGG/7/Fsm: -Slot=2; Ten-GigabitEthernet2/0/37 FSM.RX CURRENT--
Jan 19 02:31:58 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/34.send. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x6, pri=0x8000, port-index=0xef, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=4431-92d6-82de, key=0x11, pri=0x8000, port-index=0x36, state=0x3d Collector: type=3, len=16, col-max-delay=0x0 Terminator: type=0, len=0
Jan 19 02:31:58 2017 hp5700 %%10LAGG/7/Packet: -Slot=2; PACKET.Ten-GigabitEthernet2/0/37.receive. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x1, sys-mac=02a0-9895-daaf, key=0x1, pri=0x0, port-index=0x2, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x4, pri=0x8000, port-index=0xf2, state=0x3d Collector: type=3, len=16, col-max-delay=0x64 Terminator: type=0, len=0
Jan 19 02:31:59 2017 hp5700 %%10LAGG/7/Fsm: Ten-GigabitEthernet1/0/37 FSM.RX CURRENT--
Jan 19 02:31:59 2017 hp5700 %%10LAGG/7/Packet: PACKET.Ten-GigabitEthernet1/0/37.receive. size=110, subtype=1, version=1 Actor: type=1, len=20, sys-pri=0x1, sys-mac=02a0-9895-daaf, key=0x1, pri=0x0, port-index=0x1, state=0x3d Partner: type=2, len=20, sys-pri=0x8000, sys-mac=e8f7-24b9-40d3, key=0x4, pri=0x8000, port-index=0x25, state=0x3d Collector: type=3, len=16, col-max-delay=0x64 Terminator: type=0, len=0

Logs for the Cisco 6500:

Jan 19 02:31:50 cisco6500 22027920: Jan 19 02:31:48.873 CET: SP: RSTP(1010): starting topology change timer for 35 seconds
Jan 19 02:31:52 cisco6500 22027942: Jan 19 02:31:51.487 CET: SP: LACP: 
Jan 19 02:31:52 cisco6500 22027951: Jan 19 02:31:52.307 CET: SP: LACP: 
Jan 19 02:31:52 cisco6500 22027952: Jan 19 02:31:52.437 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Jan 19 02:31:52 cisco6500 22027953: Jan 19 02:31:52.437 CET: FEC: pagp_switch_is_in_port_channel: Te1/3 is part of agport
Jan 19 02:31:52 cisco6500 22027954: Jan 19 02:31:52.481 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Jan 19 02:31:52 cisco6500 22027955: Jan 19 02:31:52.481 CET: FEC: pagp_switch_is_in_port_channel: Te3/3 is part of agport
Jan 19 02:31:52 cisco6500 22027957: Jan 19 02:31:52.489 CET: %LINK-3-UPDOWN: Interface TenGigabitEthernet1/3, changed state to down
Jan 19 02:31:52 cisco6500 22027958: Jan 19 02:31:52.517 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel14, changed state to down
Jan 19 02:31:52 cisco6500 22027959: Jan 19 02:31:52.517 CET: FEC: pagp_switch_is_in_port_channel: Po14 is not part of any agport
Jan 19 02:31:52 cisco6500 22027960: Jan 19 02:31:52.525 CET: FEC: pagp_switch_portlist_in_channelgroup: Te1/3 part of group 14 
Jan 19 02:31:52 cisco6500 22027961: Jan 19 02:31:52.525 CET: FEC: pagp_switch_portlist_in_channelgroup: Te3/3 part of group 14 
Jan 19 02:31:52 cisco6500 22027962: Jan 19 02:31:52.525 CET: FEC: pagp_switch_portlist_in_channelgroup: Te3/3 part of group 14 
Jan 19 02:31:53 cisco6500 22027963: Jan 19 02:31:52.537 CET: %LINK-3-UPDOWN: Interface Port-channel14, changed state to down
Jan 19 02:31:53 cisco6500 22027964: Jan 19 02:31:52.541 CET: %LINK-3-UPDOWN: Interface TenGigabitEthernet3/3, changed state to down
Jan 19 02:31:54 cisco6500 22027965: Jan 19 02:31:52.713 CET: %PM-SPSTBY-4-ERR_DISABLE: channel-misconfig error detected on Te1/3, putting Te1/3 in err-disable state
Jan 19 02:31:54 cisco6500 22027966: Jan 19 02:31:52.737 CET: %PM-SPSTBY-4-ERR_DISABLE: channel-misconfig error detected on Te3/3, putting Te3/3 in err-disable state
Jan 19 02:31:54 cisco6500 22027967: Jan 19 02:31:53.157 CET: %PM-SPSTBY-4-ERR_DISABLE: channel-misconfig error detected on Po14, putting Te1/3 in err-disable state
Jan 19 02:31:54 cisco6500 22027968: Jan 19 02:31:53.157 CET: %PM-SPSTBY-4-ERR_DISABLE: channel-misconfig error detected on Po14, putting Te3/3 in err-disable state
Jan 19 02:31:54 cisco6500 22027969: Jan 19 02:31:53.157 CET: %PM-SPSTBY-4-ERR_DISABLE: channel-misconfig error detected on Po14, putting Po14 in err-disable state
Jan 19 02:31:54 cisco6500 22027973: Jan 19 02:31:52.431 CET: %PM-SP-4-ERR_DISABLE: channel-misconfig error detected on Po14, putting Te1/3 in err-disable state
Jan 19 02:31:54 cisco6500 22027975: Jan 19 02:31:52.475 CET: SP: LACP: if_down: Te1/3
Jan 19 02:31:54 cisco6500 22027980: Jan 19 02:31:52.475 CET: %PM-SP-4-ERR_DISABLE: channel-misconfig error detected on Po14, putting Te3/3 in err-disable state
Jan 19 02:31:54 cisco6500 22027982: Jan 19 02:31:52.527 CET: SP: LACP: if_down: Te3/3
Jan 19 02:31:54 cisco6500 22027987: Jan 19 02:31:52.527 CET: %PM-SP-4-ERR_DISABLE: channel-misconfig error detected on Po14, putting Po14 in err-disable state
Jan 19 02:31:54 cisco6500 22027988: Jan 19 02:31:52.759 CET: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet1/3, changed state to down
Jan 19 02:31:54 cisco6500 22027989: Jan 19 02:31:52.759 CET: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface TenGigabitEthernet3/3, changed state to down
Jan 19 02:31:54 cisco6500 22027990: Jan 19 02:31:52.759 CET: %LINK-SP-3-UPDOWN: Interface TenGigabitEthernet1/3, changed state to down
Jan 19 02:31:54 cisco6500 22027991: Jan 19 02:31:52.759 CET: %LINEPROTO-SP-5-UPDOWN: Line protocol on Interface Port-channel14, changed state to down
Jan 19 02:31:54 cisco6500 22027992: Jan 19 02:31:52.759 CET: %LINK-SP-3-UPDOWN: Interface Port-channel14, changed state to down
Jan 19 02:31:54 cisco6500 22027993: Jan 19 02:31:52.759 CET: %LINK-SP-3-UPDOWN: Interface TenGigabitEthernet3/3, changed state to down
Jan 19 02:31:57 cisco6500 22028027: Jan 19 02:31:56.961 CET: SP: LACP:

Thanks for your advices

lozair31 · ‎01-19-2017

network topology

peyrache · ‎01-19-2017

so now need to debug LACP packets during event i think

do you have log from switch where Server is connected ?

JY

peyrache · ‎01-19-2017

also there a know issue on Blade switch with LACP dont' know is relevant in your case

ymptom:
LACP packets received from downstream port-channel members are flooded by the 3120 switch instead of being processed.

Conditions:
Issue seen on CBS3120 switch running 12.2(40)EX1

Workaround:
Instead of LACP please use a "mode on" for bundling the member links.

Categories

Company

Local Language

Forums

Discussions

Forums

Discussions

Forums

Discussions

Forums

Discussions

Forums

Discussions

Discussions

Forums

Forums

Discussions

Forums

Discussions

Forums

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Discussion Boards

Community

Resources

Other HPE Sites

Discussions

Forums

Blogs

LACP randomly fail between hpe5700 and cisco 6509

LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509

Re: LACP randomly fail between hpe5700 and cisco 6509