5406zl ARP times out, then ping host causes link to drop ?

[Bruce Campbell_3]

We have about 20 Procurve 5406zl switches
at our site, most are layer 2/3 aggregation,
but some have hosts plugged directly in.

On at least 2 (running K.12.01 and K.12.51)
we occasionally see hosts lose their port
entry in "show arp", example 129.97.32.41 as
follows:

129.97.32.39 001422-34adc2 dynamic C4
129.97.32.41 00d0b7-59ffe9 dynamic
129.97.32.42 001372-1ece30 dynamic C10

Then, if I ping that host from my computer,
the link for that host goes down !

I 03/03/08 12:36:07 00435 ports: port B14 is Blocked by STP
I 03/03/08 12:36:09 00077 ports: port B14 is now off-line
I 03/03/08 12:36:11 00435 ports: port B14 is Blocked by STP
I 03/03/08 12:36:40 00076 ports: port B14 is now on-line

and then it shows in show arp
correctly:

129.97.32.41 00d0b7-59ffe9 dynamic B14

The users notice the drops. We have about
20,000 ports on Procurve gear at our site,
and this problem seems specific to hosts on
5406zl's which are doing the routing for
the host in question.

Any ideas ? We're opening a support case.

[Evert Goor · Accepted solution]

Please upgrade to the latest k12.57 firmware.
There are some fixes with arp in K12.52 and lower.
There is a fix PR_1000449365 : ARP & MAC tables get out of sync after a spanning tree (MSTP or RSTP) re-convergence. An ARP entry fails to be associated to the port even though the MAC entry exists. This may result in an unexpected ping failure.

Good luck

[Bruce Campbell_3]

Upgraded to K.12.57 and issue persists,
spanning tree topology hasn't changed
in 25 hours...

show arp...
129.97.164.216 001cc4-1bb520 dynamic B9
129.97.164.217 001b78-b5801d dynamic
129.97.164.218 001b78-b57cf2 dynamic C1

ping 129.97.164.217

show arp...
129.97.164.216 001cc4-1bb520 dynamic B9
129.97.164.217 001b78-b5801d dynamic A2
129.97.164.218 001b78-b57cf2 dynamic C1

show log...
I 03/05/08 08:22:56 00077 ports: port A2 is now off-line
I 03/05/08 08:22:59 00435 ports: port A2 is Blocked by STP
I 03/05/08 08:23:02 00076 ports: port A2 is now on-line

show span...

Time Since Last Change : 25 hours

show ver...

Image stamp: /sw/code/build/btm(t2g)
Dec 7 2007 14:54:57
K.12.57

We have a case open now.

[Case Van Horsen]

I've tried recreating it in my lab and I can't make it happen. The configuration of the port I'm using is completely left to default.

Can you force it to happen?

Can you post the switch configuration?

I'm using K.12.57, BTW.

casevh

[Bruce Campbell_3]

I can make it happen on 129.97.164.217 (port A2) and 129.97.164.212 (port B5) every few
minutes. The port entries on "show arp" for
them vanish every few minutes, then a ping causes the link on the port to drop and
come back up. The ports stay up between
tests, it isn't until I do the ping that
they drop and come back up. Most other hosts
on the same switch seem fine.


Config is below.


Running configuration:

; J8697A Configuration Editor; Created on release #K.12.57

hostname "mac-rt-qsn"
max-vlans 2048
time timezone -300
time daylight-time-rule User-defined begin-date 3/8 end-date 11/4
console baud-rate 9600
console inactivity-timer 5
no web-management
module 1 type J8702A
module 2 type J8702A
module 3 type J8702A
module 4 type J8702A
interface A1
name "A1"
exit
interface A2
name "A2"
exit
interface A3
name "A3"
exit
interface A4
name "A4"
exit
interface A5
name "A5"
exit
interface A6
name "A6"
exit
interface A7
name "A7-mac-ap-qsn-7--b"
exit
interface A8
name "A8"
exit
interface A9
name "A9"
exit
interface A10
name "A10"
exit
interface A11
name "A11"
exit
interface A12
name "A12"
exit
interface A13
name "A13"
exit
interface A14
name "A14"
exit
interface A15
name "A15"
exit
interface A16
name "A16"
exit
interface A17
name "A17"
exit
interface A18
name "A18"
exit
interface A19
name "A19"
exit
interface A20
name "A20"
exit
interface A21
name "A21"
exit
interface A22
name "A22"
exit
interface A23
name "A23"
exit
interface A24
name "A24"
exit
interface B1
name "A25"
exit
interface B2
name "A26"
exit
interface B3
name "A27"
exit
interface B4
name "A28"
exit
interface B5
name "A29"
exit
interface B6
name "A30"
exit
interface B7
name "A31"
exit
interface B8
name "A32"
exit
interface B9
name "A33"
exit
interface B10
name "A34"
exit
interface B11
name "A35"
exit
interface B12
name "A36"
exit
interface B13
name "A37"
exit
interface B14
name "A38"
exit
interface B15
name "A39"
exit
interface B16
name "A40"
exit
interface B17
name "A41"
exit
interface B18
name "A42"
exit
interface B19
name "A43"
exit
interface B20
name "A44"
exit
interface B21
name "A45"
exit
interface B22
name "A46"
exit
interface B23
name "A47"
exit
interface B24
name "A48"
exit
interface C1
name "C1"
exit
interface C2
name "C2"
exit
interface C3
name "C3"
exit
interface C4
name "C4"
exit
interface C5
name "C5"
exit
interface C6
name "C6"
exit
interface C7
name "C7"
exit
interface C8
name "C8"
exit
interface C9
name "C9"
exit
interface C10
name "C10"
exit
interface C11
name "C11"
exit
interface C12
name "C12"
exit
interface C13
name "C13"
exit
interface C14
name "C14"
exit
interface C15
name "C15"
exit
interface C16
name "C16"
exit
interface C17
name "C17"
exit
interface C18
name "C18"
exit
interface C19
name "C19"
exit
interface C20
name "C20"
exit
interface C21
name "C21"
exit
interface C22
name "C22"
exit
interface C23
name "C23"
exit
interface C24
name "C24"
exit
interface D1
name "C25"
exit
interface D2
name "C26"
exit
interface D3
name "C27"
exit
interface D4
name "C28"
exit
interface D5
name "C29"
exit
interface D6
name "C30"
exit
interface D7
name "C31"
exit
interface D8
name "C32"
exit
interface D9
name "B28"
exit
interface D10
name "B29"
exit
interface D11
name "B30"
exit
interface D12
name "B31"
exit
interface D13
name "B34-mac-ap-qsn-7--a"
exit
interface D14
name "C33"
exit
interface D15
name "B47"
exit
interface D16
name "B48"
exit
interface D24
name "Atria-Uplink(V395)"
speed-duplex 100-full
exit
sntp server 129.97.128.10
sntp server 129.97.129.10
ip routing
timesync sntp
sntp unicast
snmp-server community "xxxxxxx" Operator Unrestricted
snmp-server community "xxxxxxx" Unrestricted
vlan 1
name "DEFAULT_VLAN"
forbid A1-A24,B1-B24,C1-C24,D1-D24
no untagged A1-A24,B1-B24,C1-C24,D1-D24
no ip address
exit
vlan 746
name "cn-rt-mc-sci-rt-que"
untagged D24
ip address 129.97.1.182 255.255.255.252
ip igmp
exit
vlan 273
name "mac-qsn-net"
untagged A1-A6,A8-A24,B1-B24,C1-C24,D1-D12,D14-D23
ip helper-address 129.97.128.10
ip helper-address 129.97.129.10
ip address 129.97.164.193 255.255.255.192
exit
vlan 627
name "mac-net-qsn-ap-mgmt"
untagged A7,D13
ip helper-address 129.97.128.10
ip helper-address 129.97.129.10
ip address 172.16.6.17 255.255.255.240
exit
vlan 406
name "pri-mac-qsn-voip"
ip helper-address 129.97.128.10
ip helper-address 129.97.129.10
ip address 172.16.25.33 255.255.255.224
tagged A1-A6,A8-A24,B1-B24,C1-C24,D1-D12,D14-D23
exit
ip authorized-managers 129.97.108.0 255.255.255.0
ip authorized-managers 129.97.128.0 255.255.255.0
ip multicast-routing
router ospf
area 0.0.1.45
area backbone
exit
router pim
rp-address 129.97.1.5 224.0.0.0 240.0.0.0
exit
spanning-tree
spanning-tree force-version RSTP-operation
vlan 273
ip ospf 129.97.164.193 area 0.0.1.45
ip pim-sparse
ip-addr any
dr-priority 3
exit
exit
vlan 406
ip ospf 172.16.25.33 area 0.0.1.45
ip pim-sparse
ip-addr any
dr-priority 3
exit
exit
vlan 627
ip ospf 172.16.6.17 area 0.0.1.45
ip pim-sparse
ip-addr any
dr-priority 3
exit
exit
vlan 746
ip ospf 129.97.1.182 area backbone
ip ospf 129.97.1.182 dead-interval 5
ip ospf 129.97.1.182 hello-interval 1
ip ospf 129.97.1.182 priority 3
ip pim-sparse
ip-addr any
no lan-prune-delay
override-interval 500
propagation-delay 250
dr-priority 3
exit
exit

[Bruce Campbell_3]

Correction:

After each ping test, which causes the
link to down/up, the link then does a
down/up again, by itself, 106-107 seconds
later (repeatably). Then it stays up,
until I ping it again, and it does
the down/up again.

After the 106-107 seconds, the mac
disappears from "show mac" but it stays
in "show arp" (minus the port).

129.97.164.212 001cc4-66261f
129.97.164.220 001b78-b57fdd

[Bruce Campbell_3]

Starting to look like K.12.57 fixed the main
issue, and the remaining issues may be host
related. I see someone else has similar
troubles with the same printer (2605dn) we are using and seeing alot of link changes:

http://forums11.itrc.hp.com/service/forums/bizsupport/questionanswer.do?threadId=1198801

That would explain 1 of the 3 ports we're
seeing this issue on.

[Evert Goor]

try to make that port an spanningtree edge port.

[Evert Goor]

and ofcourse disable lacp on that port. Seen a lot of problems with lacp and stp with certain devices.