MSTP over L2 WAN and STP blocks

[trendchiller]

Hi !

I am experiencng a strange behaviour and need some advice.

Perhaps my config is wrong, perhaps it's not ;-)

We have 8 locations, each  location owns a 5406zlRv2-switch with tha latest firmware.

Each int A1 is connected to a L2-Infrastructure (by our ISP).

On the main-location there's a second connection (to the ISP) on int B1.

Every location has it's own VLAN and config-name and each VLAN has its own MSTP instance:

location 1: VLan 100, MSTP 1

location 2: VLan 200, MSTP 2

location 3: VLan 300, MSTP 3

location 4: VLan 400, MSTP 4

location 5: VLan 500, MSTP 5

location 6: VLan 600, MSTP 6

location 7: VLan 700, MSTP 7

location 8: VLan 800, MSTP 8

Each location knows each vlan and each MSTP-instance.

Each int A1 (,B1 in the main-location) is member ov VLAN5 (for routing-usage between the locations).

VLAN 5 is NOT member of any MSTP-instance.

VLAN 5 is OSPF-enabled.

Since the ISPs L2-Infrastructure does not support 802.1ad (LACP) until now, STP sets one interface A1 or B1 in the main-location th standby.

On the A1 (,B1) i have set up

no spanning-tree A1(,B1) bpdu-protection
no spanning-tree A1(,B1) admin-edge-port
no spanning-tree A1(,B1) auto-edge-port

Now sometimes i get

AM1: port A1,(B1) is now on-line
AM1: port A1(,B1) is Blocked by STP

in the main location.

Even when i disable one interface

int A1 disable / int B1 disable

this happens.

When this happens, OSPF loses the routes and ... you can imagine...

All ISP-connections have at least 8 MBit and the main-location has 2x 50 MBit.

So what may be wrong ?

• it it unusual / wrong to use MSTP this way over a L2 WAN ?
• should i filter bpdu packets on A1,B1 (i'd line to have STP over the locations) ?
• is it correct that each location is aware of the other MSTP instances and VLANs ?
• do the edgeports A1(,B1) [VLAN 5] have to be members in some instance ?

• What might cause the STP-blocks (sonce it's only in the main-location) ?
• Might this even be a config error by the ISP ?

• How can i manage to use both WAN-interfaces A1,B1 in the main-location actively for load-sharing (in case 802.1ad will never be enabled)
• Perhaps by enabling another VLAN on B1 and the other A1 interfaces ?

perhaps anyone might have some idea...

regards,

martin

[Vince-Whirlwind]

I doubt STP is doing anything - you get an STP message each time a port goes down and then up again.

How much bandwidth does your carrier offer to each site?

[trendchiller]

Hi !

The bandwith is in the main location 2x 50 MBit, in the remote locations it's 8 MBit each.

Mostly it seems as if this issue occurs when the link i saturated.

BUT: The ISP has QoS mapping configured on the links.

This maps HP ToS 7,6,5 to ISP ToS 5, HP ToS 4, to ISP ToS 4, HP ToS 3,2 to ISP ToS 3,2, HP ToS 1,0 to ISP ToS 0

So OSPF and STP packets should pass priorized...

[Vince-Whirlwind]

So, three things:

1- you don't need any load-sharing as your maximum bandwidth is 100Mb. (redundancy is a different issue, but you only have one switch, so maybe not a priority issue).

2- what you need is traffic-shaping - your carrier's QoS isn't going to fix dropped packets because you are trying to send more than what they've agreed to carry.

3- HP do a funny thing where by default they give IP precedence value 6 to voice and 5 to video instead of using 5 and 4. If you've got video going out at 5, you might have a problem. Can you get traffic statistics reporting that shows you how much of each traffic class your carrier is seeing?

[trendchiller]

Hi !

We cannot use the 2x 50 MBit at the same time it seems.

When both links are attached one link is blocked by STP (standby)

We have traffic shaping set up internally with DSCP values.

Every packet that enters the VLAN in the location is DSCP-tagged.

So there should not be any problem.

I also have connfigured rate-limit all out kbps 48750 on port A1 (and 7500 in the locations) as a rate limit.

I try to reach our ISP to get statistics if anything might be dropped there.

ProCurve tags OSPF and STP with ToS 7 (the ISP tells me he sees).

These are mapped to QoS 5 (max ToS class by ISP side).

We use VoIP tagged with ToS 5, Citrix tagged woth ToS 4, Print & Scan ToS 3, Management ToS 4 and bulk class 0.

So video in not (until now) an option and VoIP can rarely saturate the link because there are not so many parallel calls.

[Vince-Whirlwind]

That's odd that you can't use both links they give you (and that you pay for?!) at the same time.

If it is two separate services, it's even odder that STP kicks in when you patch both of them in at the same time. To me that sounds like you just have two patch leads coming from the carrier's simple Layer2 switch, in which case - have you tried getting 100Mb/s through a single link?

[trendchiller]

seems to be so...

the ISP uses two different physical ways for the fiber, but it terminates in the same L2-topology.

We paid for having a redundant solution with usage rights of both links az the same time.

So there's only 802.1ad (LACP) for L2 or IP-loadsharing (OSPF) for L3.

I hope they implement 802.1ad soon ;-)

100 MBit as a single link would work, but we need redundance...

[Vince-Whirlwind]

You have a single switch, so you don't really need multiple links to the carrier layer2 switch(?). The redundancy you have is 2 different WAN links.

Your core switch, link, carrier switch(assuming there is only one) are all single points of failure.

What physical device(s) have they installed on your site that you are patching your switch to?

If you want to use 2 WAN services in a properly redundant way, you either link-aggregate your connections to it, or, you install 2 routers and use a routing protocol for redundancy, which is often not going to use both links simultaneously anyway.

[trendchiller]

Until now we have a single switch.

A secone swizch will come in the near future.

At the ISP that's a big virtual infrastructure that acts as a virtual switch (seen from our side).

So i'll have a look for using OSPF for this or using LACP first...

[trendchiller]

I narrowed down the problem a bit.

when the link is saturated completely the STP and OSPF packets get lost.

So there's some problem with the network-control packets.

When i filter the bdpu-packets on the locations WAN ports then the stp blocks do not occur anymore.

So the questions arise:

When i rate-limit the interface (lets say to 50000 kbps, how should i configure the qos using int a1 bandwith-min output ?Rate-limiting uses the interface connection speed. How is bandwith-min output affected by this ?

How should VLAN5 be configured with STP that only the instance is blocked, bot the whole port on the other side ?