Sluggish network and erratic pings - 5820 IRF Core and Procurve edge

[mikedoubleu]

Hi all,

I'm hoping someone will be able to help me diagnose a problem with my new network deployment. I have been working at a college for a number of years with a completely Procurve network. Ive recently undertook a major (for us) upgrade tot he network implementing a pair of IRF'd 5820 switches at the core and then assorted Procurve switches around the edges of the network. The Procurve's are an assortment of a 5412zl, a number of 10gig LACP trunked 2920's and 2810, 2650, and 2530's.

Prior to implementing all of the recent upgrades, we had a rock solid network with low pings that never exceeded 1ms across the board. Now that I have got a good way through installing the new switching, I'm finding that staff are complaining of intermittent network connection loss, slow performance, and I am seeing pings approaching 50ms, but only every x number of pings.

For example, if I ping our 5412zl switch which is connected to the core with an LACP trunk of two 10gig links, I am getting pings like this...

36ms

1ms

1ms

1ms

1ms

3ms

7ms

19ms

2ms

1ms

1ms

1ms

1ms

I can forgive the first high ping, as I think this tends to be an arp cache update causing the first delay, and see it in in a number of places, but the pattern of 4-7 1ms pings and then a few high pings, and the 4-7 1ms pings is worrying and I think indicative of a problem.

The things I have tried so far:

I have shutdown all links on the IRF 5820 core switches and then brought them up one by one to try and elimnate network loops

I have turned off one of the core switches to see if that would help

I have converted all bridge aggregation interfaces from static to dynamic and back again

I have firmware updated the core switches and top of rack 2920's, then disconnected everything else

I am at this stage fairly certain that a network loop is NOT the cause of the problem, but I seem to have hit a limit in my knowledge and am not not sure where to go with diagnosing the problem?

I hope somebody may be able to help, and thank you or taking the time to read this.

I have included the configs from the two main switches in question below.

Thanks again

Mike

Here is the code for the 5412zl:

SWI-LIBRARY# sh conf

Startup configuration: 58

; J8698A Configuration Editor; Created on release #K.15.14.0007
; Ver #05:18.7f.ff.3f.ef:4d
hostname "SWI-LIBRARY"
module 1 type j8706a
module 2 type j9538a
module 4 type one-06
module 5 type j8702a
module 6 type j8702a
module 7 type j8702a
module 8 type j8702a
module 9 type j8702a
module 10 type j8702a
trunk E1-E4,H19-H20 trk1 trunk
trunk H21-H22 trk2 trunk
trunk H23-H24 trk3 trunk
trunk B1-B2 trk8 lacp
timesync sntp
sntp unicast
sntp server priority 1 172.16.80.11
ip default-gateway 10.4.200.0
interface H20
   disable
   exit
interface H22
   disable
   exit
interface H24
   disable
   exit
snmp-server community "public" unrestricted
snmp-server contact "Mike Wroughton" location "PCRM5"
vlan 1
   name "MANAGEMENT"
   no untagged D1i,E5-E24,F1-F24,G1-G24,H1-H18,I1-I24,J1-J24
   untagged A1-A24,B3-B8,D2i-D3i,Trk1-Trk3,Trk8
   ip address 10.4.200.4 255.0.0.0
   ip igmp
   forbid E5-E24,F1-F9,F12-F24,I1-I24,J3-J24
   exit
vlan 3
   name "PUBLIC"
   no ip address
   ip igmp
   forbid A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   exit
vlan 5
   name "DMZ"
   no ip address
   ip igmp
   forbid A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   exit
vlan 12
   name "BLUESKY"
   tagged A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   no ip address
   ip igmp
   exit
vlan 14
   name "WIFI"
   tagged A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   no ip address
   ip igmp
   exit
vlan 16
   name "SERVERS"
   untagged E5-E24
   no ip address
   ip igmp
   forbid A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   exit
vlan 17
   name "STAFFWIFI"
   tagged A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   no ip address
   ip igmp
   exit
vlan 18
   name "STUDENTWIFI"
   tagged A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   no ip address
   ip igmp
   exit
vlan 19
   name "GUESTWIFI"
   tagged A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   no ip address
   ip igmp
   exit
vlan 20
   name "ADMIN"
   tagged A1-A24,B3-B8,D2i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   no ip address
   ip igmp
   exit
vlan 21
   name "ACADEMIC"
   untagged G1-G24,H1-H18,I1-I24,J1-J24
   tagged A1-A24,B3-B8,D2i,Trk1-Trk3,Trk8
   no ip address
   ip igmp
   exit
vlan 30
   name "VOICE"
   tagged A1-A24,B3-B8,D1i-D3i,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   no ip address
   ip igmp
   exit
vlan 70
   name "PROTOCOL"
   no ip address
   ip igmp
   exit
vlan 110
   name "ISCSI1"
   untagged F1-F24
   no ip address
   forbid A1-A24,B3-B8,E5-E24,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   jumbo
   exit
vlan 120
   name "ISCSI2"
   no ip address
   forbid A1-A24,B3-B8,E5-E24,F1-F24,G1-G24,H1-H18,I1-I24,J1-J24,Trk1-Trk3,Trk8
   jumbo
   exit
spanning-tree
spanning-tree Trk1 priority 4
spanning-tree Trk2 priority 4
spanning-tree Trk3 priority 4
spanning-tree Trk8 priority 4
spanning-tree config-name "NCSLAN"
no autorun
no dhcp config-file-update
no dhcp image-file-update
password manager

[mikedoubleu]

Code for the 5820's

 sysname HP
#
 dhcp relay server-group 1 ip 172.16.80.13
 dhcp relay server-group 1 ip 172.16.80.14
 dhcp relay server-group 1 ip 172.16.80.61
 dhcp relay server-group 1 ip 172.16.90.14
#
 irf domain 324
 irf mac-address persistent timer
 irf auto-update enable
 undo irf link-delay
 irf member 1 priority 32
#
 domain default enable system
#
 dns server 172.16.80.11
#
 ipv6
#
 telnet server enable
#
 multicast routing-enable
#
 password-recovery enable
#
igmp-snooping
#
vlan 1
 name MANAGEMENT
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 3
 name PUBLIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 5
 name DMZ
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 12
 name BLUESKY
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 14
 name WIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 16
 name SERVERS
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 17
 name STAFFWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 18
 name STUDENTWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 19
 name GUESTWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 20
 name ADMIN
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 21
 name ACADEMIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 22
 name MUSIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 30
 name VOICE
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 70
 name Protocol
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 110
 name iSCSI1
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 120
 name iSCSI2
 igmp-snooping enable
 igmp-snooping version 3
#
domain system
 access-limit disable
 state active
 idle-cut disable
 self-service-url disable
#
user-group system
 group-attribute allow-guest
#
local-user manager
 password cipher $c$3$odTL4+uij2fDoXvPRafuuZlISEsDtQfhW+wZ
 authorization-attribute level 3
 service-type ssh
#
 stp instance 0 priority 4096
 stp enable
#
cwmp
 undo cwmp enable
#
interface Bridge-Aggregation3
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation4
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation5
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation6
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation7
 description Link to Server Switch Stack
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation8
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation9
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation10
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation11
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation12
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation13
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation14
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation15
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation16
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation17
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation18
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation19
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation20
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation21
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation22
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation23
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation24
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation25
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation26
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation27
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation28
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation67
#
interface NULL0
#
interface Vlan-interface1
 ip address 10.4.200.0 255.0.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface12
 ip address 172.12.0.10 255.255.0.0
 dhcp select relay
 proxy-arp enable
#
interface Vlan-interface14
 ip address 172.14.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface16
 ip address 172.16.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface17
 ip address 172.17.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface18
 ip address 172.18.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface19
 ip address 172.19.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface20
 ip address 172.20.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface21
 ip address 172.21.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface22
 ip address 172.22.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface30
 ip address 172.30.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface70
 ip address 172.70.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface GigabitEthernet1/0/25
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 25
#
interface GigabitEthernet1/0/26
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 26
#
interface GigabitEthernet1/0/27
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 27
#
interface GigabitEthernet1/0/28
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 28
#
interface GigabitEthernet2/0/25
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 25
#
interface GigabitEthernet2/0/26
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 26
#
interface GigabitEthernet2/0/27
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 27
#
interface GigabitEthernet2/0/28
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 28
#
interface M-GigabitEthernet0/0/0
#
interface Ten-GigabitEthernet1/0/3
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 3
#
interface Ten-GigabitEthernet1/0/4
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 4
#
interface Ten-GigabitEthernet1/0/5
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 5
#
interface Ten-GigabitEthernet1/0/6
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 6
#
interface Ten-GigabitEthernet1/0/7
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 7
#
interface Ten-GigabitEthernet1/0/8
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 8
#
interface Ten-GigabitEthernet1/0/9
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 9
#
interface Ten-GigabitEthernet1/0/10
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 10
#
interface Ten-GigabitEthernet1/0/11
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 11
#
interface Ten-GigabitEthernet1/0/12
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 12
#
interface Ten-GigabitEthernet1/0/13
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 13
#
interface Ten-GigabitEthernet1/0/14
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 14
#
interface Ten-GigabitEthernet1/0/15
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 15
#
interface Ten-GigabitEthernet1/0/16
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 16
#
interface Ten-GigabitEthernet1/0/17
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
#
interface Ten-GigabitEthernet1/0/18
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 18
#
interface Ten-GigabitEthernet1/0/19
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 19
#
interface Ten-GigabitEthernet1/0/20
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 20
#
interface Ten-GigabitEthernet1/0/21
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 21
#
interface Ten-GigabitEthernet1/0/22
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 22
#
interface Ten-GigabitEthernet1/0/23
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 23
#
interface Ten-GigabitEthernet1/0/24
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 24
#
interface Ten-GigabitEthernet2/0/3
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 3
#
interface Ten-GigabitEthernet2/0/4
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 4
#
interface Ten-GigabitEthernet2/0/5
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 5
#
interface Ten-GigabitEthernet2/0/6
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 6
#
interface Ten-GigabitEthernet2/0/7
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 7
#
interface Ten-GigabitEthernet2/0/8
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 8
#
interface Ten-GigabitEthernet2/0/9
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 9
#
interface Ten-GigabitEthernet2/0/10
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 10
#
interface Ten-GigabitEthernet2/0/11
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 11
#
interface Ten-GigabitEthernet2/0/12
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 12
#
interface Ten-GigabitEthernet2/0/13
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 13
#
interface Ten-GigabitEthernet2/0/14
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
#
interface Ten-GigabitEthernet2/0/15
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 15
#
interface Ten-GigabitEthernet2/0/16
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 16
#
interface Ten-GigabitEthernet2/0/17
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
#
interface Ten-GigabitEthernet2/0/18
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 18
#
interface Ten-GigabitEthernet2/0/19
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 19
#
interface Ten-GigabitEthernet2/0/20
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 20
#
interface Ten-GigabitEthernet2/0/21
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 21
#
interface Ten-GigabitEthernet2/0/22
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 22
#
interface Ten-GigabitEthernet2/0/23
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 23
#
interface Ten-GigabitEthernet2/0/24
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 24
#
interface Ten-GigabitEthernet1/0/1
#
interface Ten-GigabitEthernet1/0/2
#
interface Ten-GigabitEthernet2/0/1
#
interface Ten-GigabitEthernet2/0/2
#
igmp
#
 ip route-static 0.0.0.0 0.0.0.0 172.16.101.254
 ip route-static 10.22.240.0 255.255.240.0 172.16.150.60
 ip route-static 172.12.0.0 255.255.0.0 172.16.0.254
 ip route-static 172.16.0.0 255.255.0.0 10.4.200.0
 ip route-static 172.34.0.0 255.255.0.0 172.16.0.3
 ip route-static 192.168.115.0 255.255.255.0 172.16.150.60
 ip route-static 192.168.123.0 255.255.255.0 172.16.0.254
#
 dhcp enable
#
 ssh server enable
#
 load xml-configuration
#
 load tr069-configuration
#
user-interface aux 0 1
user-interface vty 0 15
 authentication-mode scheme
 protocol inbound ssh
#
irf-port 1/1
 port group interface Ten-GigabitEthernet1/0/1 mode enhanced
 port group interface Ten-GigabitEthernet1/0/2 mode enhanced
#
irf-port 2/2
 port group interface Ten-GigabitEthernet2/0/1 mode enhanced
 port group interface Ten-GigabitEthernet2/0/2 mode enhanced
#
return

[mikedoubleu]

And here is the config for the 5820's:

 version 5.20.105, Release 1808P25
#
 sysname HP
#
 dhcp relay server-group 1 ip 172.16.80.13
 dhcp relay server-group 1 ip 172.16.80.14
 dhcp relay server-group 1 ip 172.16.80.61
 dhcp relay server-group 1 ip 172.16.90.14
#
 irf domain 324
 irf mac-address persistent timer
 irf auto-update enable
 undo irf link-delay
 irf member 1 priority 32
#
 domain default enable system
#
 dns server 172.16.80.11
#
 ipv6
#
 telnet server enable
#
 multicast routing-enable
#
 password-recovery enable
#
igmp-snooping
#
vlan 1
 name MANAGEMENT
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 3
 name PUBLIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 5
 name DMZ
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 12
 name BLUESKY
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 14
 name WIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 16
 name SERVERS
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 17
 name STAFFWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 18
 name STUDENTWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 19
 name GUESTWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 20
 name ADMIN
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 21
 name ACADEMIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 22
 name MUSIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 30
 name VOICE
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 70
 name Protocol
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 110
 name iSCSI1
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 120
 name iSCSI2
 igmp-snooping enable
 igmp-snooping version 3
#
domain system
 access-limit disable
 state active
 idle-cut disable
 self-service-url disable
#
user-group system
 group-attribute allow-guest
#
local-user manager
 password cipher $c$3$odTL4+uij2fDoXvPRafuuZlISEsDtQfhW+wZ
 authorization-attribute level 3
 service-type ssh
#
 stp instance 0 priority 4096
 stp enable
#
cwmp
 undo cwmp enable
#
interface Bridge-Aggregation3
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation4
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation5
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation6
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation7
 description Link to Server Switch Stack
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation8
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation9
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation10
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation11
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation12
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation13
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation14
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation15
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation16
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation17
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation18
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation19
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation20
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation21
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation22
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation23
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation24
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation25
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation26
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation27
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation28
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation67
#
interface NULL0
#
interface Vlan-interface1
 ip address 10.4.200.0 255.0.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface12
 ip address 172.12.0.10 255.255.0.0
 dhcp select relay
 proxy-arp enable
#
interface Vlan-interface14
 ip address 172.14.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface16
 ip address 172.16.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface17
 ip address 172.17.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface18
 ip address 172.18.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface19
 ip address 172.19.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface20
 ip address 172.20.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface21
 ip address 172.21.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface22
 ip address 172.22.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface30
 ip address 172.30.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface70
 ip address 172.70.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface GigabitEthernet1/0/25
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 25
#
interface GigabitEthernet1/0/26
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 26
#
interface GigabitEthernet1/0/27
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 27
#
interface GigabitEthernet1/0/28
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 28
#
interface GigabitEthernet2/0/25
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 25
#
interface GigabitEthernet2/0/26
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 26
#
interface GigabitEthernet2/0/27
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 27
#
interface GigabitEthernet2/0/28
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 28
#
interface M-GigabitEthernet0/0/0
#
interface Ten-GigabitEthernet1/0/3
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 3
#
interface Ten-GigabitEthernet1/0/4
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 4
#
interface Ten-GigabitEthernet1/0/5
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 5
#
interface Ten-GigabitEthernet1/0/6
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 6
#
interface Ten-GigabitEthernet1/0/7
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 7
#
interface Ten-GigabitEthernet1/0/8
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 8
#
interface Ten-GigabitEthernet1/0/9
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 9
#
interface Ten-GigabitEthernet1/0/10
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 10
#
interface Ten-GigabitEthernet1/0/11
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 11
#
interface Ten-GigabitEthernet1/0/12
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 12
#
interface Ten-GigabitEthernet1/0/13
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 13
#
interface Ten-GigabitEthernet1/0/14
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 14
#
interface Ten-GigabitEthernet1/0/15
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 15
#
interface Ten-GigabitEthernet1/0/16
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 16
#
interface Ten-GigabitEthernet1/0/17
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
#
interface Ten-GigabitEthernet1/0/18
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 18
#
interface Ten-GigabitEthernet1/0/19
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 19
#
interface Ten-GigabitEthernet1/0/20
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 20
#
interface Ten-GigabitEthernet1/0/21
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 21
#
interface Ten-GigabitEthernet1/0/22
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 22
#
interface Ten-GigabitEthernet1/0/23
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 23
#
interface Ten-GigabitEthernet1/0/24
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 24
#
interface Ten-GigabitEthernet2/0/3
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 3
#
interface Ten-GigabitEthernet2/0/4
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 4
#
interface Ten-GigabitEthernet2/0/5
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 5
#
interface Ten-GigabitEthernet2/0/6
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 6
#
interface Ten-GigabitEthernet2/0/7
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 7
#
interface Ten-GigabitEthernet2/0/8
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 8
#
interface Ten-GigabitEthernet2/0/9
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 9
#
interface Ten-GigabitEthernet2/0/10
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 10
#
interface Ten-GigabitEthernet2/0/11
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 11
#
interface Ten-GigabitEthernet2/0/12
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 12
#
interface Ten-GigabitEthernet2/0/13
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 13
#
interface Ten-GigabitEthernet2/0/14
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
#
interface Ten-GigabitEthernet2/0/15
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 15
#
interface Ten-GigabitEthernet2/0/16
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 16
#
interface Ten-GigabitEthernet2/0/17
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
#
interface Ten-GigabitEthernet2/0/18
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 18
#
interface Ten-GigabitEthernet2/0/19
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 19
#
interface Ten-GigabitEthernet2/0/20
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 20
#
interface Ten-GigabitEthernet2/0/21
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 21
#
interface Ten-GigabitEthernet2/0/22
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 22
#
interface Ten-GigabitEthernet2/0/23
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 23
#
interface Ten-GigabitEthernet2/0/24
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 port link-aggregation group 24
#
interface Ten-GigabitEthernet1/0/1
#
interface Ten-GigabitEthernet1/0/2
#
interface Ten-GigabitEthernet2/0/1
#
interface Ten-GigabitEthernet2/0/2
#
igmp
#
 ip route-static 0.0.0.0 0.0.0.0 172.16.101.254
 ip route-static 10.22.240.0 255.255.240.0 172.16.150.60
 ip route-static 172.12.0.0 255.255.0.0 172.16.0.254
 ip route-static 172.16.0.0 255.255.0.0 10.4.200.0
 ip route-static 172.34.0.0 255.255.0.0 172.16.0.3
 ip route-static 192.168.115.0 255.255.255.0 172.16.150.60
 ip route-static 192.168.123.0 255.255.255.0 172.16.0.254
#
 dhcp enable
#
 ssh server enable
#
 load xml-configuration
#
 load tr069-configuration
#
user-interface aux 0 1
user-interface vty 0 15
 authentication-mode scheme
 protocol inbound ssh
#
irf-port 1/1
 port group interface Ten-GigabitEthernet1/0/1 mode enhanced
 port group interface Ten-GigabitEthernet1/0/2 mode enhanced
#
irf-port 2/2
 port group interface Ten-GigabitEthernet2/0/1 mode enhanced
 port group interface Ten-GigabitEthernet2/0/2 mode enhanced
#
return

[Vince-Whirlwind]

What is your spanning-tree design/topology? You should check out your physical topology and make sure you give your "Core" switch the lowest STP priority. If you have additional daisy-chaining of switches, lower or raise spanning-tree priorities so that the switch closer to the "Core" always has a lower priority than the switch that daisy-chains off it.

As far as losses of connetivity go, do you monitor your network devices & links, using IMC/Solarwinds/etc...?

Also, you should check all your switch logs and see what's going on.

Check CPU over time (but with most granular time-period possible) to see if there are any odd spikes.

[mikedoubleu]

Hi Vince-Whirwind,

Thanks for the feedback. I have implemented your recommendation of changing the stp priority on the switches in the college. It unfortunately hasnt resolved the problem though.

We have the 2 core switches in an IRF cluster, and then approximately 10 switches directly connected to the cluster via LACP connections over CAT6 or Fibre depending on location. There are then approximately another 20 switches around the college connected downstream from those ten switches.

We are using MSTP but we are not utilizing multiple instances at this time.

I cant see anything of any relevance in the log files I've looked at so far.

We havent got IMC up and running as yet, Its being installed on Tuesday :-)

I have included the config for the 5820 here, because it appears to have disappeared from above, I've snipped some of the interfaces out, but they are all configured the same:

Thanks

Mike

#
 version 5.20.105, Release 1808P25
#
 sysname HP
#
 dhcp relay server-group 1 ip 172.16.80.13
 dhcp relay server-group 1 ip 172.16.80.14
 dhcp relay server-group 1 ip 172.16.80.61
 dhcp relay server-group 1 ip 172.16.90.14
#
 irf domain 324
 irf mac-address persistent timer
 irf auto-update enable
 undo irf link-delay
 irf member 1 priority 32
#
 domain default enable system
#
 dns server 172.16.80.11
#
 ipv6
#
 telnet server enable
#
 multicast routing-enable
#
 password-recovery enable
#
igmp-snooping
#
vlan 1
 name MANAGEMENT
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 3
 name PUBLIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 5
 name DMZ
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 12
 name BLUESKY
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 14
 name WIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 16
 name SERVERS
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 17
 name STAFFWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 18
 name STUDENTWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 19
 name GUESTWIFI
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 20
 name ADMIN
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 21
 name ACADEMIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 22
 name MUSIC
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 30
 name VOICE
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 70
 name Protocol
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 110
 name iSCSI1
 igmp-snooping enable
 igmp-snooping version 3
#
vlan 120
 name iSCSI2
 igmp-snooping enable
 igmp-snooping version 3
#
domain system
 access-limit disable
 state active
 idle-cut disable
 self-service-url disable
#
user-group system
 group-attribute allow-guest
#
local-user manager
 password cipher $c$3$odTL4+uij2fDoXvPRafuuZlISEsDtQfhW+wZ
 authorization-attribute level 3
 service-type ssh
#
 stp instance 0 priority 4096
 stp enable
#
cwmp
 undo cwmp enable
#
interface Bridge-Aggregation3
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation4
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation5
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation6
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation7
 description Link to Server Switch Stack
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation8
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation9
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation10
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation11
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation12
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation13
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation14
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation15
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation16
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation17
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation18
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation19
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation20
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation21
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation22
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation23
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation24
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation25
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation26
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation27
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation28
 port link-type trunk
 port trunk permit vlan all
#
interface Bridge-Aggregation67
#
interface NULL0
#
interface Vlan-interface1
 ip address 10.4.200.0 255.0.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface12
 ip address 172.12.0.10 255.255.0.0
 dhcp select relay
 proxy-arp enable
#
interface Vlan-interface14
 ip address 172.14.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface16
 ip address 172.16.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface17
 ip address 172.17.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface18
 ip address 172.18.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface19
 ip address 172.19.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface20
 ip address 172.20.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface21
 ip address 172.21.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface22
 ip address 172.22.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface30
 ip address 172.30.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface Vlan-interface70
 ip address 172.70.0.1 255.255.0.0
 dhcp select relay
 dhcp relay server-select 1
 proxy-arp enable
#
interface GigabitEthernet1/0/25
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 undo jumboframe enable
 port link-aggregation group 25
#
interface GigabitEthernet1/0/26
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 undo jumboframe enable
 port link-aggregation group 26
#
interface GigabitEthernet1/0/27
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 undo jumboframe enable
 port link-aggregation group 27
#
interface GigabitEthernet1/0/28
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 undo jumboframe enable
 port link-aggregation group 28
#
interface GigabitEthernet2/0/25
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 port link-aggregation group 25
#
interface GigabitEthernet2/0/26
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 port link-aggregation group 26
#
interface GigabitEthernet2/0/27
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 port link-aggregation group 27
#
interface GigabitEthernet2/0/28
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 port link-aggregation group 28
#
interface M-GigabitEthernet0/0/0
#
interface Ten-GigabitEthernet1/0/3
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 undo jumboframe enable
 port link-aggregation group 3
#
interface Ten-GigabitEthernet2/0/24
 port link-mode bridge
 port link-type trunk
 port trunk permit vlan all
 broadcast-suppression 50
 multicast-suppression 50
 undo jumboframe enable
 port link-aggregation group 24
#
interface Ten-GigabitEthernet1/0/1
#
interface Ten-GigabitEthernet1/0/2
#
interface Ten-GigabitEthernet2/0/1
#
interface Ten-GigabitEthernet2/0/2
#
igmp
#
 ip route-static 0.0.0.0 0.0.0.0 172.16.101.254
 ip route-static 10.22.240.0 255.255.240.0 172.16.150.60
 ip route-static 172.12.0.0 255.255.0.0 172.16.0.254
 ip route-static 172.16.0.0 255.255.0.0 10.4.200.0
 ip route-static 172.34.0.0 255.255.0.0 172.16.0.3
 ip route-static 192.168.115.0 255.255.255.0 172.16.150.60
 ip route-static 192.168.123.0 255.255.255.0 172.16.0.254
#
 dhcp enable
#
 ssh server enable
#
 load xml-configuration
#
 load tr069-configuration
#
user-interface aux 0 1
user-interface vty 0 15
 authentication-mode scheme
 protocol inbound ssh
#
irf-port 1/1
 port group interface Ten-GigabitEthernet1/0/1 mode enhanced
 port group interface Ten-GigabitEthernet1/0/2 mode enhanced
#
irf-port 2/2
 port group interface Ten-GigabitEthernet2/0/1 mode enhanced
 port group interface Ten-GigabitEthernet2/0/2 mode enhanced
#
return

[Vince-Whirlwind]

Until you have your network monitoring up and running, you'll have to do it manually:

Get on each device and monitor CPU, memory, and all ports utilisation. Watch for spikes in any of those.

Configure all your devices to log severity warning, or try info. Use a syslog server, ideally. Look for weird events/messages.

Update all your firmware versions.

Don't worry too much about ping response times - the CPU deliberately puts these on hold sometimes, as it has the lowest of all priorities.

Get a better idea of what the users are actually complaining about - DHCP relay/snooping on these switches is an utter pain sometimes.

[sdide]

Hi Mike

I think pinging the switch itself might be misleading. Ping is going to the CPU. If the switch is busy with something else, ping response might be slow even tho traffic on the ASICs is forwarded fine.

So ping tests should be done to non-network units , servers og IPSLA units or something similar. That said - the staff complaining - can't be ignored.

Can you make a small drawing of where you get bad performance, how is the topology, what units are pinging, etc.

Regards Søren.

[e-hpe]

Hello,

What was the resolution to this issue?

I am running IRF'd HPE 5510's at the core and seeing high jitter when pinging both the interface ip and vlan ip address.

I have performed complete isolation testing and factory reset the switch only configuring the IP Address and still have the same issue.

[e-hpe]

As per HP this is expected behaviour;

https://support.hpe.com/hpsc/doc/public/display?docId=emr_na-c03251146

[parnassus]

I think the OP solved his network issue back in 2014...even if this thread deserved more investigation (as example: trunks/BAGGs configurations were - IMHO - not so consistents with regard to Non Trunk/LACP selections on peers' ends).

As reference for future visitors (and reflecting informations provided in 2014):

HP 5412zl side:

On HP ProCurve 5412zl (J8689A) - which didn't act as the core - we see FOUR trunks (LAGs in Comware terminology):

1. trunk E1-E4,H19-H20 trk1 trunk <--- 4 (Module 4) plus 2 (Module 8) for a grant total of SIX 1Gbps ports aggregated into a group "trk1" set as non protocol (so NON LACP IEEE 802.3ad)
2. trunk H21-H22 trk2 trunk <--- 2 (Module 8) for a grant total of TWO 1Gbps ports aggregated into a group "trk2" set as non protocol (so NON LACP IEEE 802.3ad)
3. trunk H23-H24 trk3 trunk <--- 2 (Module 8) for a grant total of TWO 1Gbps ports aggregated into a group "trk3" set as non protocol (so NON LACP IEEE 802.3ad)
4. trunk B1-B2 trk8 lacp <--- 2 (Module 2) for a grant total of TWO 10Gbps ports aggregated into a group "trk8" set as LACP IEEE 802.3ad

But then:

interface H20
   disable
   exit
interface H22
   disable
   exit
interface H24
   disable
   exit

shows us that you disabled one link for each non 10G trunk, so trk1, trk2 and trk3 run respectively with 4+1, 1 and 1 active links each one (instead of 4+2, 2 and 2). Pretty strange.

Note how trk8 is an untagged member of VLAN 1 and it is a tagged member of VLAN 3, 5, 12, 14, 16, 17, 18 ,19, 20, 21, 30 (excluding VLAN 70, 110 and 120); the same can be said about trk1, trk2 and trk3.

IRF side

It's highly probable that trk8's links (B1 and B2 member ports) coterminate into one (of total twenty-four the IRF has defined [*]) of IRF's defined LAG since its member links are 10G SFP+ interfaces (and IRF has bascially only mostly 10G capable interfaces): no LAG is defined as LACP on that side. Pretty strange

LAG 3 = 1/0/3+2/0/3
LAG 4 = 1/0/4+2/0/4
LAG 5 = 1/0/5+2/0/5
LAG 6 = 1/0/6+2/0/6
LAG 7 = 1/0/7+2/0/7
LAG 8 = 1/0/8+2/0/8
LAG 9 = 1/0/9+2/0/9
LAG 10 = 1/0/10+2/0/10
LAG 11 = 1/0/11+2/0/11
LAG 12 = 1/0/12+2/0/12
LAG 13 = 1/0/13+2/0/13
LAG 15 = 1/0/15+2/0/15
LAG 16 = 1/0/16+2/0/16
LAG 18 = 1/0/18+2/0/18
LAG 19 = 1/0/19+2/0/19
LAG 20 = 1/0/20+2/0/20
LAG 21 = 1/0/21+2/0/21
LAG 22 = 1/0/22+2/0/22
LAG 23 = 1/0/23+2/0/23
LAG 24 = 1/0/24+2/0/24
LAG 25 = 1/0/25+2/0/25
LAG 26 = 1/0/26+2/0/26
LAG 27 = 1/0/27+2/0/27
LAG 28 = 1/0/28+2/0/28

• Note how 1/0/14, 1/0/17, 2/0/14 and 2/0/17 aren't part of any LAG.
• Note LAG (Bridge-Aggregation) 17 and 67 defined has logical entities, not popluated with any member link.
• Note all LAGs permits (transport) all VLANs defined into IRF (including VLAN 22, 70, 110 and 120).
• Note how VLAN 22 is not defined into HP ProCurve 5412zl.
  
  IRF's STP is enabled (stp enable) and its priority is not explicitly set to highest level (4096 is not 0) as it should be indeed the stp instance 0 priority 4096 configuration.

So...at that time...more should have been written...