Network STP recalculation and intermittent connection

[Glenn Joseph Andal]

Hi,

Can someone help me on our long overdue problem with one of our biggest client?

We are experiencing very erratic behaviour of network infrastructure sometime core switches hangs, sometime ospf is missing. and this morning they are not accessible but all switches are up. Here is our initial findings, please verify.

Check STP topology change (Libis Core Switch)

Topology change occurred in 11:45

2. Ping/access other core switches' IP addresses from Libis Core Switch. All interfaces status are up.
a. unable to ping GCC Core Switch backbone IP (192.168.2.1)
b. able to access PDC primary switch backbone IP (192.168.33.3)
c. able to access PDC backup switch backbone IP (192.168.33.2)

3. Ping/access GCC Core switch from PDC primary switch

a. unable to ping GCC Core Switch backbone IP (192.168.3.1)

4. Check OSPF routes.

OSPF routes are okay.

5. After several minutes, unable to access PDC core switches from Libis Core Switch

6. From SMG network, able to ping GCC Firewall and GCC Switch but encountered with several timeouts.

7. After 30 mins, STP topology has no change but still can't access PDC and GCC core switch.

Our assumption is a loop broadcast occurred in the network. Initial workaround was to disable port A1 in Libis core switch (Libis - GCC link).

Action items:

1. Identify what triggers the topology change.
2. Check why STP normalizes for 30 mins and still can't access PDC and GCC core switches

[Pieter 't Hart]

are you sure this is a live log ?
I 01/01/90 00:00:01 00413 SNTP: updated time by -60 seconds

if time is updated by sntp how come "/90" ?
How should we match lines in the different logs?
If there is a bogus ntp packet received, then this can cause timers to get confused.

Apart from this it could be a firmware issue related to time counter.