Need help connecting 4 ProCurve 3500yl switches together

[Schicki1]

Hi everyone,

I'm trying to link 4 3500yl switches together so all devices connected to each switch can talk to each other. I'm trying to avoid daisy chaining, so when one switch goes down, it won't cause disruption for other switches.

Given switches A,B,C,D, would the best way to connect be:

A->B, A-C, A->D, B->C, B->D, C->D  and then run RSTP on all of the switches to avoid loops and flooding?

Or would you do a different setup like this?

Get two more switches E and F

Design:

E<->F (trunk)

Then connect A->E, A->F, B->E, B->F, C->E, C->F, D->E, D->F   and run RSTP on every switch. So switches A,B,C,D would be able to talk to each other through their connection on switch E, but if switch E goes down, link to switch F will become active and they can all talk to each other through switch F.

Any other ideas? Comments? How else would you do it, or which way do you prefer?

Regards,

Sam

[paulgear]

Hi Sam,

This is really a question of your requirements: what are the most important aspects of your network? Cost? Redundancy? Performance?

In each case you'll need turn on (M|R)STP.

1. If you want a simple solution that provides redundancy at minimum cost (i.e. minimum number of ports used), then just connect the switches in a ring. If any one switch or any one link fails, each remaining switch can still connect to the others.
2. If you need a high level of redundancy, the configuration you suggested with each switch interconnected to every other switch would probably be best.
3. If you want redundancy and extra performance, consider a ring where each switch connects to its two neighbours with an LACP trunk.
4. Of course, #3 comes at the cost of using extra ports.  So you might to make one of the inter-switch links a single port.  STP would normally choose this port to be idle, since it's lower bandwidth than the trunks.
5. If you wanted maximum redundancy and performance, you could do a four-switch interconnect with LACP trunks.
6. By implementing #5, you'd use a minimum of 6 ports per switch on interconnects, some of which may be idle most of the time. So you might decide to do something similar to #4 and make some links single ports.  However this is a more complex solution, which means it's a little harder to troubleshoot and support.

All of these are legitimate options.  It's really up to you to decide what you're trying to achieve and find the right balance between redundancy, performance, cost, and complexity.

[Schicki1]

Hi Paul,

thanks for your answer!

You are actually right with #5 it costs a lot of ports. So if I went with my suggested setup #2, having two core switches and connect the edge switches to each core, could I get better performance by making the connection to the core switch be a trunk of (2) ports? That way I would utilize 4 ports per switch instead of 6. 

Say I made a 2 port trunk to core1 and called it trk1 and I made a 2 port trunk to core 2 and called it trk2 from the same edge switch, I assume trk2 would get turned "off" due to STP, but trk1 should be functioning. Or does STP not allow trunks? Will STP turn off a link of a 2 port trunk?

What do you think?

Thanks,

Sam

[paulgear]

Hi Sam,

STP views trunks as single links, so if your 2-port trunks are equal distance from the root bridge, one of them will likely be disabled (use "show spanning-tree" to see which one).  This makes it a bit of a waste having a trunk.

If it were me and performance weren't my first criteria (which it usually isn't - redundancy is typically of greater concern for me), i would use a ring rather than a mesh, and make one part of the ring two individual links rather than a two-port trunk.  Then STP will chose those links to be idle rather than the trunks, due to their lower individual speeds.  (See diagram - the thick lines are trunks, the thin lines are individual links.)

An alternative if you really want to have a mesh is to have half of your network (say A-B, A-C, A-D) use trunks, and the other half (B-C, B-D, C-D) use individual links so that the A side is automatically prioritised by STP.  Make sure you set the right switch to be the root bridge (A in both of these cases).

Hope that makes sense.

[Schicki1]

Hi Paul,

so I drew a map to visualize my planned setup a bit better. In the map, both black and red lines are 2 port LACP trunks to each core switch. Everything should be working together through core 1, and if core 1 dies, then fail over to core 2. Core 1 is the root.

Are you saying that even in that case the 2 port LACP trunk going from edge 1 to core 1 will have one of the ports in the LACP trunk get disabled, so it will only be 1 port trunk between core 1 and edge 1? Why is that? Or did I misunderstand something?

My assumption was all the red LACP trunks would actually get disabled to core 2, which is normal and also desireable.

Thanks,

Sam

[paulgear]

I'm not sure on the exact interaction between LACP & STP, but i expect that if you lost one of the links in the LACP trunk, it would cause the link speed to change, which might cause a spanning tree reconvergence.  I think you'd end up having the same topology, but there may be some issues with convergence times.  (I'd have to lab this out and go back over my notes to confirm.)

Also, with the setup you've diagrammed, you'll end up with core 1 -> core 2 traffic going over the edge switches, which i would consider undesirable.  It would be desirable to add a core 1 -> core 2 LACP trunk as well.

[Schicki1]

Paul,

I actually don't need any traffic between core1 and core2 switches, those were introduced just to connect the edge switches together and to the router.

Can you explain briefly how this is different from your ring solution? In your ring solution you had LACP trunks as well, wouldn't a down link cause one of the links of the LACP trunk to become slow and cause reconvergence?

I would really appreciate if you have a lab environment to test this. I'm planning this so I can buy the switches, that's why I can't test myself...

Thanks,

Sam

[Richard Litchfield]

If you are thinking about buying additional devices, maybe the 3800 mesh/stack switches would be the best option? very similar to the E/F diag, but E/F looks like a single device, so no messy complexity, and the red/black lines are aggregated with LACP so you also double your bandwidth (no spanning tree blocking, although you should still have a loop-avoidance mechanism).

[paulgear]

Hi Sam,

There will still be traffic between core1 & 2 regardless, so you should plan for it.

The difference between this and the ring (or a stack - see below) is that it requires two whole switches in order to do the job of 2-4 ports per edge device, and it doesn't save any edge ports.

I don't have a lab environment big enough to test the whole scenario - i was merely thinking to test an LACP link between two switches and confirm or deny the STP reconvergence issue. Richard may be able to answer this off the top of his head.

I like Richard's idea of 3800s instead. Or if you prefer Comware, 5500-HI/EI/SI can do the same.

If you haven't actually bought the switches yet, i would take a step back and have another think about the problems you're trying to solve. My gut tells me there are other more important design issues to address.

[Schicki1]

Hi Richard,

do the 3800s support a special configuration? I haven't been able to find something. I though stacking was related to ease of management of multiple switches and not anything else?
Will the 3800s help me expand the network beyond those two switches when we grow?
Also those 3800s seem quite expensive. :(

Let me know what you think.

Thanks for your ideas!

Sam

[Schicki1]

Paul,

thanks for taking your time as well answering my questions!

When you say there will still be traffic between core1 and 2, do you mean because of the traffic from the edge switches that actually might not all go through core1, but sometimes go through core2? Otherwise I don't get why core1 and 2 would be communicating?

What is the worst case for STP reconvergence in case of a link down in LACP trunk? 10-15 seconds block on the port?

So the advantage of 3800s in your opinion would be because of reconvergence issue? Because those can be configured as mesh?

Thanks,
Sam

[paulgear]

Hi Sam,

There will always be small amounts of management traffic to/from core2, e.g. NTP, SNMP, ssh, etc. It has to get there somehow. This happens via the edge switches if you don't have a link from core1 to core2.

If you're going to buy these specifically for the purpose of connecting the edge switches, then there's no reason not to connect them together - it gives core2 a better picture of what happens when core1 goes down, and will result in faster convergence times for the edges, because each edge switch can simply switch to its alternate port.

I can't remember off the top of my head what the convergence figures are for RSTP, but they are definitely faster if there's a pre-existing alternate path to the root. (Assuming the use of RSTP, which i think is the default nowadays.)

With 3800s and 5500s you can use true stacking, so there's no STP required to link them together.

[Schicki1]

Hey guys,

so I'm still not done with the setup and saw that the 3500s now support DT (distirbuted trunking) for switches. Would it make sense to go the route of DT and connect all edge switches like that to the core switches 1 and 2?

Thanks,

Sam