topic Re: STP loops in Switches, Hubs, and Modems

STP loops

procurvenewbee — Sun, 23 Sep 2007 17:06:01 GMT

Hi All,

I am in real need of some help promptly.

Here is the rundown:

1. Core A & Core B 5406, running ospf (as DR on VLAN 10 and BDR on VLAN 20 for core A and vice versa for CoreB, via ip ospf priority 255 & 254). The two cores are to be connected via a L2 trunk (not done yet as it results into Spanning tree loops). The core switches are running RSTP (plan to set it to MSTP but not done yet as couple of L2 closets are still running 4000 switches). Core switches have vlan 10, vlan 20, few other vlans for L2 closet termination and vlan 100 as management vlan. The L2 trunk between them has vlan 10, vlan 20 as tagged (believe vlan 100 was not included but I will add it as well over this L2 trunk). The links on these 5400s, going over to the 3500 switches as below, have both vlan 10 & 20 tagged (do not know if this will matter, but I plan to tag only one VLAN on each of these link corresponding to the one set up as tagged at the other end 3500). These links have bpdu-filter configured at each end so that all such links becomes forwarding, to take advantage of L3 ospf ECMP. Do not want to have spanning tree block paths as then ECMP load balancing will not work.

2. Most of the closets have two 3500 L3 switches set up as distribution switches running ospf with each of the core A & B. 3500 A is connected to core A over VLAN 10 and to core B over VLAN 20 and 3500 B connected to core A over vlan 20 and core B over vlan 10. ip ospf priority 0 so that these switches do not participate in election to become Dr/BDR. There is L2 trunk between the two 3500s in each of these closets, with all vlans other than vlan 10 & 20 tagged. Vlan 10 & 20 are forbidden so as to avoid any loop thru the core switches. The unplinks from these distribution switches over to the core switches have vlan 10 or vlan 20 tagged (not both). Vlan 1 by default will also be leaking to core switches. 3500s have mstp set up (5400s have rstp and if I connect L2 trunk between the two cores, a loop forms, because of all forwarding links).

3. Each closet with two L3 3500s have 10 to 20, L2 2650 switches and they all run mstp within closet (ach closet is set up as separate mstp region). Each floor switch is uplinked to both 3500s. mstp runs fine within closet and there is VLAN load balanced over the two uplinks from the floor switches.

4. The distribution switches within each closet are set up for vrrp and it works fine as well. However because of absence of the L2 trunk (for the time being) between core A & B and because of strange procurve ospf ECMP implentation which does not load balance per packet or per destination but only per destination network, some of the routes are learned via Core A and others via Core B ( really sequentially where first, third, fifth subnet routes will be learned via one core and second, fourth, sixth subnet routes via second core), and servers and internet gateway being presently only connected to Core A, there results blackhole in reaching to some of the subnets. If ospf load balancing was like in Cisco, then each subnet route as installed in each 3500 will have two entries in their routing table, one via Core A & other via Core B and we could influence the desired path as well. But I am not able to obtain this functionality. Procurve is in the process of implementing refined ECMP but that may be months away.

5. I believe my problems can be resolved if I implement MSTP on both core switches instead of RSTP, (will add all vlans throughout the network, not assigning any ports (other than tagging all vlans on links going to all 3500s), but then it becomes L2/L3 instead of only L3 route point to point links between distribution switches and the core switches. I believe MSTP config on core switches will cause automatic fallback to STP for L2 closets with 4000 switches if I change RSTP to MSTP on core switches.

Please advise as to the best configuration for Core switches so as to avoid spanning tree loops so that L2 trunk between them can be connected and blackholing of traffic be eliminated.

Appreciate help

Thanks

Re: STP loops

Matt Hobbs — Sun, 23 Sep 2007 17:29:04 GMT

Can you attach a network map and the configs you have so far?

Re: STP loops

procurvenewbee — Sun, 23 Sep 2007 18:16:16 GMT

Thanks Matt. Here is the network map. I do not have access to the network now and I can send you configs tomorrow.

Re: STP loops

Olaf Borowski — Tue, 25 Sep 2007 10:48:57 GMT

Hi,

Looking at your diagram, there is an MSTP config problem. If you want a number of switches to be in the same region, 3 parameters have to match on each switch:
1: name
2: version number
3: instance to vlan mapping.

#3 is the problem. The instance to vlan mapping is not consistant on all switches in the same region. Therefore, the switches become their own region (example: green boxes, trunk has vlan 1,16,116,100, other links have 1,16,100,10,20).

This might not be the orgininal problem, but fix one problem at a time.

Verify that MSTP is the way you want it first.

Re: STP loops

procurvenewbee — Wed, 26 Sep 2007 12:24:58 GMT

Sorry for late response. I have been out sick.

I am sure my MSTP configuration is correct and within each region (each L3 closet of two L3 switches and L2 floor switches) all three parameters are identical. Please note that I am not running MSTP with Core switch. So I do not see any MSTP configuration issue.

Matt, since I am not in office and I do not have remote access to this network, I have not been able to post the configs. I will try to do so tonight. Please review and advise. I am also not sure as to how will I be able to run MSTP between my L3 closet switches (3524s) and L3 core switches as all of these then have to be made part of same region. My preference will be to not run STP/MSTP and keep it all L3 from distribution up.
Could you advise meanwhile based on these inputs and the network map?

Thanks

Re: STP loops

procurvenewbee — Thu, 27 Sep 2007 12:45:03 GMT

Matt,
I have finally been able to get to work and here are the reprentative configurations for your review and advice.

And Olaf, unless I misunderstood, I have purposly forbidden vlan 10 & 20 on the L2 trunk between the distribution switches. You will find from the configuratations below that all instances have been correctly mapped to the required vlans. I have floor 18 switches ( L2 & L3 dist as well as two cores below).

1. 3524 Distribution switch (18FLRDSA) relevant config:

interface 19-20 speed-duplex auto-1000

interface 21,23 speed-duplex 1000-full

interface 21 name "To Core A port A1"

interface 23 name "To Core B port A2"
interface 1-18 name "To FloorSWPort49"

interface 19-20 name "Trk1 To 18FLRDSB"

trunk 19-20 Trk1 Trunk
vlan 1

tagged 1-18,Trk1

no ip address

exit

vlan 18

name "FLR-18 Data"

ip helper-address 10.10.116.20

ip address 10.10.18.1/24

tagged 1-18,Trk1

vlan 118

name "FLR-18 Voice"

ip helper-address 10.10.116.20

ip address 172.20.18.1/24

tagged 1-18,Trk1

ip router-id 192.168.18.1

router ospf

area backbone

redistribute connected

vlan 10

name OSPF10

ip address 10.10.10.3/24

tag 21,1-18

forbid trk1

ip ospf area 0

ip ospf priority 0

ip ospf hello-interval 1

ip ospf dead-interval 2

vlan 20

name OSPF20

ip address 10.10.20.3/24

tag 23,1-18

forbid trk1

ip ospf area 0

ip ospf priority 0

ip ospf hello-interval 1

ip ospf dead-interval 2

vlan 100

name Mgmt

ip address 192.168.18.1/24

tagged 1-18,Trk1

primary-vlan 100

spanning-tree force-version mstp

spanning-tree

spanning-tree 21,23 bpdu-filter

spanning-tree trap errant-bpdu

spanning-tree config-name FLR18

spanning-tree config-revision 18

spanning-tree instance 1 vlan 18 100

spanning-tree instance 1 priority 0

spanning-tree instance 2 vlan 118

spanning-tree instance 2 priority 1

spanning-tree priority 0

router vrrp

vlan 18

vrrp vrid 18

owner

virtual-ip-address 10.10.18.1/24

enable

vlan 118

vrrp vrid 118

owner

virtual-ip-address 172.20.18.1/24

enable

vlan 100

vrrp vrid 100

owner
virtual-ip-address 192.168.18.1/24

enable

2. 3524 Distribution switch (18FLRDSB) relevant config:

interface 19-20 speed-duplex auto-1000

interface 21,23 speed-duplex 1000-full

interface 21 name "To Core A port A2"

interface 23 name "To Core B port A1"
interface 1-18 name "To FloorSWPort50"

interface 19-20 name "Trk1 To 18FLRDSA"

trunk 19-20 Trk1 Trunk
vlan 1

tagged 1-18,Trk1

no ip address

vlan 18

name "FLR-18 Data"

ip helper-address 10.10.116.20

ip address 10.10.18.2/24

tagged 1-18,Trk1

vlan 118

name "FLR-18 Voice"

ip helper-address 10.10.116.20

ip address 172.20.18.2/24

tagged 1-18,Trk1

ip router-id 192.168.18.2

router ospf

area backbone

redistribute connected

vlan 10

name OSPF10

ip address 10.10.10.4/24

tag 23,1-18

forbid trk1

ip ospf area 0

ip ospf priority 0

ip ospf hello-interval 1

ip ospf dead-interval 2

exit

vlan 20

name OSPF20

ip address 10.10.20.4/24

tag 21,1-18

forbid trk1

ip ospf area 0

ip ospf priority 0

ip ospf hello-interval 1

ip ospf dead-interval 2

vlan 100

name Mgmt

ip address 192.168.18.2/24

tagged 1-18,Trk1

primary-vlan 100

spanning-tree force-version mstp

spanning-tree

spanning-tree 21,23 bpdu-filter

spanning-tree trap errant-bpdu

spanning-tree config-name FLR18

spanning-tree config-revision 18

spanning-tree instance 1 vlan 18 100

spanning-tree instance 1 priority 1

spanning-tree instance 2 vlan 118

spanning-tree instance 2 priority 0

spanning-tree priority 1

router vrrp

vlan 18

vrrp vrid 18

backup

virtual-ip-address 10.10.18.2/24

enable

vlan 118

vrrp vrid 118

backup

virtual-ip-address 172.20.18.2/24

enable

vlan 100

vrrp vrid 100

backup

virtual-ip-address 192.168.18.2/24

enable

3. Core A

interface A1 name "fiber link to FLR18 3524A port 21" speed-duplex 1000-full
interface A2 name "fiber link to FLR18 3524B port 23" speed-duplex 1000-full
interface A9 name "fiber link to Core B port A9" speed-duplex 1000-full
vlan 1
name "DEFAULT_VLAN"
untagged A9,B1-B24,
no untagged A10-A24

vlan 100
name "MGMT"
untagged A10-A24
ip address 192.168.100.1 255.255.255.0
tagged A9

vlan 10
name "OSPF10"
ip address 10.10.10.1 255.255.255.0
tagged A9,A1

vlan 20
name "OSPF20"
ip address 10.10.20.1 255.255.255.0
tagged A9,A2

primary-vlan 100
ip router-id 192.168.100.1

router ospf
area backbone
redistribute connected
exit
spanning-tree
spanning-tree priority 0 force-version RSTP-operation
spanning-tree A1-A2 bpdu-filter

vlan 10
ip ospf 10.10.10.1 area backbone
ip ospf 10.10.10.1 dead-interval 2
ip ospf 10.10.10.1 hello-interval 1
ip ospf 10.10.10.1 priority 255

vlan 20
ip ospf 10.10.20.1 area backbone
ip ospf 10.10.20.1 dead-interval 2
ip ospf 10.10.20.1 hello-interval 1
ip ospf 10.10.20.1 priority 254
exit

4. Core B

interface A1 name "fiber link to FLR18 3524A port 23" speed-duplex 1000-full
interface A2 name "fiber link to FLR18 3524B port 21" speed-duplex 1000-full
interface A9 name "fiber link to Core A port A9" speed-duplex 1000-full
vlan 1
name "DEFAULT_VLAN"
untagged A9,B1-B24,
no untagged A10-A24

vlan 100
name "MGMT"
untagged A10-A24
ip address 192.168.100.2 255.255.255.0
tagged A9

vlan 10
name "OSPF10"
ip address 10.10.10.2 255.255.255.0
tagged A9,A2

vlan 20
name "OSPF20"
ip address 10.10.20.2 255.255.255.0
tagged A9,A1

primary-vlan 100
ip router-id 192.168.100.2

router ospf
area backbone
redistribute connected
exit
spanning-tree
spanning-tree priority 1 force-version RSTP-operation
spanning-tree A1-A2 bpdu-filter

vlan 10
ip ospf 10.10.10.2 area backbone
ip ospf 10.10.10.2 dead-interval 2
ip ospf 10.10.10.2 hello-interval 1
ip ospf 10.10.10.2 priority 254

vlan 20
ip ospf 10.10.20.2 area backbone
ip ospf 10.10.20.2 dead-interval 2
ip ospf 10.10.20.2 hello-interval 1
ip ospf 10.10.20.2 priority 255
exit

5. Typical L2 Floor Switch 2650 config

interface 49-50 speed-duplex auto-1000

interface 49 name "To 18FLRDSA"

interface 50 name "To 18FLRDSB"

vlan 100

name Mgmt

ip address 192.168.18.10/24

tagged 49-50

exit

vlan 10

no ip add

tag 49-50

exit

vlan 20

no ip add

tag 49-50

vlan 1

no ip add

tag 49-50

vlan 18

no ip address

tag 49-50

vlan 118

no ip address

tag 49-50

primary-vlan 100

dhcp-snooping

dhcp-snooping vlan 18 118

interface 49-50 dhcp-snooping trust

spanning-tree protocol-version mstp

spanning-tree config-name FLR18

spanning-tree config-revision 18

spanning-tree instance 1 vlan 18 100

spanning-tree instance 2 vlan 118

loop-protect 1-48

loop-protect disable-timer 60

loop-protect trap loop-detected

Re: STP loops

Matt Hobbs — Sun, 30 Sep 2007 09:25:14 GMT

It's a lot of data to look through, so I'm just going to comment on the network map.

It's obvious there there is a loop there in VLAN 10 and 20 when you bring up that link (possibly also VLAN1).

I don't understand why VLAN 10 and 20 needs to go all the way to your edge switches.

I would keep the Core and Distribution switches on completely different VLAN ID's to that of the edge. I would even go a step further and make sure all links between the core are point-to-point, untagged only in their own VLANs.

Re: STP loops

procurvenewbee — Sun, 30 Sep 2007 11:16:32 GMT

Thanks Matt.

VLAN 10 & 20 are only between distribution and core switches (I named them ospf10 & ospf 20). Since these vlan IDs are on distribution switches within each closet / MSTP region, I was told that for MSTP working, I need to create these VLAN IDs on edge switches as well and these should be tagged on the uplinks between edge and distribution switches else digest will not match and MSTP vlan load balancing will not work. You can see that no ports or IP addresses were assigned in the edge switches for these VLANs.

I believe VLAN 1, I had tagged on one side so that it is isolated off the uplink between core and distribution switches.

So do you want me to remove vlan 10 & vlan 20 ids from edge switches?

You will also notice that I have tried to keep VLAN100 ID for management but this has different subnet for each closet/distribution region and core also has this but again different subnet. I do not have vlan 100 tagged on the links between distribution and core switches.

Please advise.

Appreciate excellent support.

Re: STP loops

procurvenewbee — Sun, 30 Sep 2007 11:25:37 GMT

Matt,

Further to my post above, I forgot to add that I have set up the links between the distribution to the core as suggested by you to be point to point with only one required vlan ID.

VLAN 1 is by default untagged on the core sw side of the link to distribution switch, but on distribution switch side, VLAN 1 is tagged. That should isolate and remove any chance of VLAN 1 becoming a STP loop issue.

Please review and advise.

Thanks

Re: STP loops

procurvenewbee — Sun, 30 Sep 2007 15:09:14 GMT

Also I am not sure if my keeping span priority 0 and 1 on the two MSTP distribution switches (one of them will become CIST regional root), will cause any issues with Core switches which have RSTP and span prio 0 and 1 on them. I had assumed that by using bpdu-filter on the links between cores and distribution switches, I am disabling STP and that all links are thus forwarding and there will be no loop based on forbidding the vlan 10 & 20 on the layer 2 trunk between Dist switches and hence all links forwarding will allow ECMP OSPF to load balance. But I was not considering that vlan 10 & 20 created on edge switches (which as I said above I created for MSTP functionality) will create loop if this is so. I am awaiting response if VLAN 10 & 20 should be removed from edge switches and that only VLAN IDs mapped to instances have to be on the edge switches.

Thanks