HP 5406zl Lockup Lights Solid

[PerryLund]

I have an HP switch 5406zl with PoE modules. The unit locks up the network randomly (several times a week). All the active ports lights (mode / link) are solid with no blinking. Most of the time reboot the switch clears the problem. On occasion it boots to a lock mode immediately.

Any help with the problems.

[parnassus]

Hi, please provide all possible details to help diagnose the issue your Switch is experiencing (as example: Switch's logs, Switch's configuration/running configuration, these ones sanitized outputs - I mean: remove any serial numbers or any other details you feel should not be released to public - of various typical CLI show based commands...related to flash, software version, installed modules, poe statuses...).

Have you checked your network for any physical (sometime: user generated) loop? is Power Supply OK? what's about Switch's log (CLI)? What is your network topology? was there any change in the Switch's configuration lately?

Help the HPE Community to help you...provide info: it's quite hard to guess what is happening (or what happened) to your Switch by knowing just a LED status description.

[PerryLund]

Our network has this HP core switch with 2 fiber runs to two building wings. There is a 3Com Superstack 4400 at one end, and a TP-Link switch at the other end. The TP-Link connects to an Engenius POE switch via copper Ethernet. The core switch has a couple Dell servers, an Aruba wireless switch, and a couple routers attached. I will attach a diagram. 

The lockup problem was present about every 3-4 weeks a year ago, then became a weekly occurrence. At times we rebooted twice a day, then we go for a week without an issue. A 3Com switch failed last week, and was replaced with the Engenius switch.

Here is the running configuration file.

Running configuration:

; J8697A Configuration Editor; Created on release #K.16.02.0014
; Ver #0f:01.81.30.02.34.59.2c.6b.ff.f7.fc.7f.ff.3f.ef:33
hostname "HP-E5406zl"
module 1 type j8702a
module 2 type j8705a
module 3 type j8702a
module 4 type j8702a
module 5 type j8702a
module 6 type j8702a
power-over-ethernet pre-std-detect ports A1-A24,B1-B20,D1-D24
qos protocol ip priority 7
qos type-of-service ip-precedence
timesync sntp
sntp unicast
sntp 30
sntp server priority 1 207.32.21.253
time daylight-time-rule continental-us-and-canada
time timezone -480
web-management ssl
web-management idle-timeout 6000
ip default-gateway 172.16.0.105
interface A1
name "POE Phone"
exit
interface A2
name "POE Phone"
exit
interface A3
name "POE Phone"
exit
interface A4
name "POE Phone"
exit
interface A5
name "POE Phone"
exit
interface A6
name "POE Phone"
exit
interface A7
name "POE Phone"
exit
interface A8
name "POE Phone"
exit
interface A13
name "SonicWall"
exit
interface A15
name "PaloAlto"
exit
interface B1
name "Peter - Dell Server"
exit
interface B2
name "Peter - Dell Server"
exit
interface B3
name "Peter - Dell Server"
exit
interface B4
name "Peter - Dell Server"
exit
interface B5
name "Peter - Dell Server"
exit
interface B6
name "Peter - Dell Server"
exit
interface B7
name "Paul - Dell Server"
exit
interface B8
name "Paul - Dell Server"
exit
interface B9
name "Paul - Dell Server"
exit
interface B10
name "Paul - Dell Server"
exit
interface B11
name "Paul - Dell Server"
exit
interface B12
name "Paul - Dell Server"
exit
interface B13
name "Engenius Switch"
exit
interface B14
name "Aruba Wireless"
exit
interface B15
name "Mystery White"
exit
interface B16
name "Public Address"
exit
interface B20
name "Phone System"
exit
interface B23
flow-control
name "West Wing"
speed-duplex 1000-full
exit
interface B24
flow-control
name "North Admin"
exit
interface D18
name "POE Phone - Where"
exit
interface D24
name "POE Phone - Where"
exit
snmp-server community "public" unrestricted
snmp-server host 172.16.1.99 community "public" trap-level not-info
no snmp-server enable traps dhcp-server
snmp-server contact "Perry Lund / Gary Ide" location "ELHS Main Telco Room"
vlan 1
name "DEFAULT_VLAN"
untagged A1-A24,B1-B24,C1-C24,D1-D24,E1-E24,F1-F24
ip address 172.16.0.5 255.255.0.0
exit
spanning-tree
no spanning-tree bpdu-throttle
spanning-tree mode rapid-pvst
spanning-tree force-version rstp-operation
no tftp server
no autorun
no dhcp config-file-update
no dhcp image-file-update
device-profile name "default-ap-profile"
cos 0
exit
password manager

[PerryLund]

The only warning log messages are the following:

I 02/18/17 01:57:45 00076 ports: port F21 is now on-line
I 02/18/17 01:58:13 00077 ports: port E2 is now off-line
I 02/18/17 01:58:15 00076 ports: port E2 is now on-line
I 02/18/17 01:58:21 00077 ports: port E22 is now off-line
I 02/18/17 01:58:23 00076 ports: port E22 is now on-line
I 02/18/17 01:58:26 00077 ports: port E2 is now off-line
I 02/18/17 01:58:29 00076 ports: port E2 is now on-line
I 02/18/17 01:58:33 00077 ports: port E8 is now off-line
I 02/18/17 01:58:35 00076 ports: port E8 is now on-line
I 02/18/17 01:59:16 00077 ports: port F17 is now off-line
I 02/18/17 01:59:17 00077 ports: port F10 is now off-line
I 02/18/17 01:59:18 00076 ports: port F17 is now on-line
I 02/18/17 01:59:21 00076 ports: port F10 is now on-line
I 02/18/17 01:59:28 00077 ports: port F17 is now off-line
I 02/18/17 01:59:31 00076 ports: port F17 is now on-line

The end point connections are to Dell and HP desktop PCs in a computer lab. The desktop PCs are running Windows 10 Pro and all have gigabit Ethernet (onboard motherboard). Dell Vostro 200 series and HP dc7900 series PCs.

[Vince-Whirlwind]

Those log lines look like a bunch of PCs coming up simultaneously, as if something has just been switched on?

Did the lockup happen immediately after those lines were logged?

Can you still log into the switch when it's "locked up"?

If so, what does a show lldp neighbor tell you at that time?

What about CPU and MEM stats?

What about port traffic stats?

Do you have STP, BPDU guard and loop-protect configured on all the switchports that are pacthed to all those ports that come up immediately prior to a lockup?

Do you have a performance monitoring tool such as Solarwinds?