Switches, Hubs, and Modems
Showing results for 
Search instead for 
Did you mean: 

Please validate my 3 layer design

Frequent Advisor

Please validate my 3 layer design

Hi All,

I am attaching a visio diagram of proposed network. Please review and let me know if all looks good. I have been careful in planning so that there will be no loop. There are multiple closets with VLAN numbering on each floor to have a separate ID for voice and data and then multiple such floor switches (2600) will connect to two 3500 distribution switches within each closet. Each closet containing these two 3500 and multiple 2600 will run MSTP and will be a unique region. Then all such closet 3500 switches will uplink to two core switches (5400) and there is ospf between distribution and core layer. So spanning tree is disabled with bpdu filtering on these uplinks (though only one VLAN on these links, but tagged for qos purposes). Since core has to connect to a switch consolidating phone servers and we did not want to run mstp there, core is running RSTP between the phone server switch.

I wanted to be sure that this arrangement will not create any loop especially thru ospf uplinks where there is no spannning tree running (and it is my understanding that stoping bpdu packets will eliminate any chance of causing loop thru these routed links) and we want to make sure that all links are forwarding here for load balancing of routes (ospf ecmp).

Please add your comments / update the visio if there seems to be issues that will cause spanning tree loops. While we can run MSTP on phone switch side if required, but we did not see much need to do load balancing as one gig link forwarding is more than enough for phone server. Especially please do look into the vlan ID tags on the uplinks and confirm that they are okay for MSTP / RSTP purposes.

Thanks a lot for all your help.
Frequent Advisor

Re: Please validate my 3 layer design

Dear Experts,

I am still looking for some advice on this. If you will please look at my diagram, I do have confusion on the trunk set up between the 3500 switches. I am required to have all VLANs tagged to satisfy MSTP requirements, but then it will also result into a loop from one 3500 to core and back to the second 3500 and then over to first 3550 over this inter 3500 trunk as these all links carry vlan 2 (and vlan 3).

My plan was that if I have all these uplinks forwarding, ospf will be able to utilize all for load balancing and I will also have more bandwidth available from the closet going over to the core, but if this will result into a loop, I will have to remove bpdu filtering from these uplinks and add something like spanning-tree A2 port-cost 30000, where A2 is fiber port going to 3500B and likewise on 5400B on A1 port going to 3500A.

Please help me on this as I have to implement it in next couple of days.

Full points will be awarded to anyone who will provide me the solution.


Frequent Advisor

Re: Please validate my 3 layer design

in my understanding OSPF is layer3 routing protocol, while you are worried about layer2 loops. I would not do bpdu filtering inside of my network. Better to use it on edge ports, where desktops are connected, or foreign network connects. We had a problem with client network propagating their spanning tree root into our network and disrupting topology, that's why we enabled bpdu filtering on that edge port.
Connection between closet and 5400 cores, I would configure both 2 and 3 VID on each of 4 links.
Frequent Advisor

Re: Please validate my 3 layer design

Thanks for your reply.

My concern is that even on these L3 ospf links, with 5400 running RSTP, there can be a loop with potential broadcast traffic over these ospf links completing a physical loop thru core switches and distribution switches. My diagram was little wrong and it should be read as having VLAN 3 tagged on the cross links btewen 3500 and 5400s and VLAN 2 tagged on straight links. So now even if I assume that brodcast will stop at VLAN boundry, I still see a loop forming through 3500A to 5400A to 5400B to 3500B and back to 3500A.

And if I still assume physical link (independent of VLANs) forming loop, then I also may have loop thru 3500A to 5400A to 3500B and back to 3500A. Likewise on 3500B to 5400B to 3500A and back to 3500B.

I think possible solution can be like this:

1. Remove vlan 2 and 3 from L2 trunk between 3500s. This will not upset MSTP digest as these vlans are not assigned to any instance.

2. limit broadcast between two core switches and also over the uplinks to be 1%.

3. Still keep bpdu-filter on uplinks to not have RSTP kick in on these routed links (RSTP otherwise will see its BPDUs cross vlan boundries during initial start up of switch and execution of its start up configuration).

4. Since no spanning tree is configured to run on these uplink ports (RSTP disabled by bpdu-filter), all these ports will stay up. Any chance for a packet to loop back to its originating switch will not happen as there will now be a physical discontinuity in the VLANs being propagated thru the physical loops.

5. Broadcast limiting will further mitigate any such concerns as there is no expectation of too much broadcast traffic on these uplinks (only for ARP, no dhcp, no bpdu etc). 1% limit on 1Gbps port with 100Mbps max broadcast traffic allowed by design, becomes only 1Mbps then.

Please review and provide your comments.

Pieter 't Hart
Honored Contributor

Re: Please validate my 3 layer design

I have not read your drawing in detail.
I think you are trying to do too much yourself!
Especially when your knowledge is limited you better leave this up to the "intelligence" built into the switches.

spanning-tree loops is not something you need to avoid at all costs.
The spanning-tree is juist the algorith you need so the switches themselves decide wich path to block when a loop occurs. If another path becomes unavailable the previously blocked patch is opened and used as an alternative.

First when using MSTP you must not confuse physical loops with spanning-tree loops.
With MSTP the same physical link can be forwarding one vlan and blocking another.
The algorithm is designed to determine the topology for each vlan separately.
Loops are automatically broken by puting one ore more (if needed) link(s) in blocked state per vlan.

For this to work, you better not (definitely must not!) use bpdu-filtering on any link between switches!!!

only use bpdu filtering on links to devices wich you do not want to participate in your spanning-tree topology (edge ports, hosts).
so bpdu packets from unknown sources cannot interfere with your topology.

I think that for what you want, you only need to configure spanning-tree (primary/secondary) root(s) on a specific switch (for each vlan).

Frequent Advisor

Re: Please validate my 3 layer design

I validated my design myself today and tested that all my assumptions are correct. So yes, you can turn off spanning tree (with bpdu filter) selectively on the routed L3 ports betweeb 3500s and 5400s and all links remain forwarding without any loop. You just have to make sure that there is no vlan continuity across any phsical loop path.

Since no one was able to validate / confirm my design, but two folks did their best to help, I am going to award 5 points to each of them.

Thanks a lot.
André Beck
Honored Contributor

Re: Please validate my 3 layer design


I haven't looked at the drawing (you should supply it in a more generally viewable format, like a PNG), but from reading what you describe, you are on the right way.

Given that HP still doesn't support routed interfaces (in anything < 9xxx), using dedicated transit VLANs on single ports with BPDU filters and IP transit networks configured on their SVIs is the only way to build a pure L3 interswitch link. As long as there is exactly one transit VLAN per ISL and those links are established through access ports (single untagged VLAN, I assume a single tagged VLAN should make no difference but I have not tested this), you will *not* establish any loops. With these links in the OSPF backbone area and the usual topology (V-topology, triangles not squares etc) every distribution switch will establish two ECMP routes to the core and vice versa. You should also follow the other best practices (use loopback interfaces for the router-id, try to keep the access networks passive, summarize them if there are a lot of them, keep the VRRP master with the STP root on the distributions etc).

BTW, it's not the BPDU filters that prevent the loops. In fact, BPDU filters can create loops (they stop STP from doing its job of preventing them). What prevents the loops is the topology, with every single VLAN going exactly point to point. There cannot be any loop this way. The BPDU filters prevent Single STP from establishing a false combined topology that would unnecessarily block all but one of the uplinks (with true per-VLAN Spanning Tree the filters would be unnecessary, but of course platforms doing real PVST do also have routed interfaces in the first place).

BTW, why does your core still have L2 connections at all? It should be pure L3 and every L3-L2-demarcation should be at the distribution layer (or something equivalent to the distribution/access building block, like a services block or the server farm).

For an L3 switched core design, this is the way to go with ProCurve yl/zl devices. Been there, done that. A number of compromises to accept here and there, but basically it works. ProCurve hardware (< 9xxx class) seemingly is still designed with the campus spanning VLANs design in mind.

Frequent Advisor

Re: Please validate my 3 layer design

Thanks Andre for your reponse. Yes I fully understand that bpdu filtering is to kill STP on the required uplinks ( I still have to have STP on the core for terminating other switches and to guard against other unforseen loops caused. So yes bpdu filter can cause loops if not used sensibly.

My distribution to core uses L2 uplinks with STP stopped so that I can use all links (all forwarding) to use best ECMP.

Yes loopbacks were just added in procurve provsion switches and I will use those as router id for ospf.

Thanks again.