How to configure and do cabling for BFD MAD on 4 IRF switches

[pradeep7739]

Hi Team,

I am trying to build a irf stack with 4 switches, how to configure BFD MAD for this, i know for 2 switches, but i am confused for 4 switches.

should i do cabling from master to all 3 slave switches? or should they be daisy chained 

[parnassus]

BFD MAD without an assist device (a fifth Switch) will require you to directly link any IRF Member with any other IRF member so...four IRF Members will require 6 links (three dedicated interfaces on each IRF Member)...to save interfaces you can deploy BFD MAD with an assist device (the fifth Switch) and so link, on each IRF Member, just one port to that assist device...using only  4 links (so four interfaces, one for each IRF Member).

[pradeep7739]

if i understood your answer, i am looking at above cabling for BFD MAD.

How about configuration, will all the port be in same VLAN? 

[parnassus]

Yes, BFD MAD dedicated ports must belong to the same VLAD dedicated to MAD.

Something like (four IRF Members directly connectet each others using three interfaces on each IRF Member, no BFD MAD intermediate device):

# IRF BFD MAD dedicated VLAN Id

vlan 200
description IRF-BFD-MAD

# IRF BFD MAD VLAN Id SVIs defined for each IRF Member (here we used a /28 Subnet for up to 14 hosts)

interface Vlan-interface 200
mad bfd enable
mad ip address 10.0.255.1 255.255.255.240 member 1
mad ip address 10.0.255.2 255.255.255.240 member 2
mad ip address 10.0.255.3 255.255.255.240 member 3
mad ip address 10.0.255.4 255.255.255.240 member 4

# IRF Member 1

interface GigabitEthernet 1/0/1
port link-mode bridge
port access vlan 200
stp disable

interface GigabitEthernet 1/0/2
port link-mode bridge
port access vlan 200
stp disable

interface GigabitEthernet 1/0/3
port link-mode bridge
port access vlan 200
stp disable

# IRF Member 2

interface GigabitEthernet 2/0/1
port link-mode bridge
port access vlan 200
stp disable

interface GigabitEthernet 2/0/2
port link-mode bridge
port access vlan 200
stp disable

interface GigabitEthernet 2/0/3
port link-mode bridge
port access vlan 200
stp disable

# IRF Member 3

interface GigabitEthernet 3/0/1
port link-mode bridge
port access vlan 200
stp disable

interface GigabitEthernet 3/0/2
port link-mode bridge
port access vlan 200
stp disable

interface GigabitEthernet 3/0/3
port link-mode bridge
port access vlan 200
stp disable

# IRF Member 4

interface GigabitEthernet 4/0/1
port link-mode bridge
port access vlan 200
stp disable

interface GigabitEthernet 4/0/2
port link-mode bridge
port access vlan 200
stp disable

interface GigabitEthernet 4/0/3
port link-mode bridge
port access vlan 200
stp disable

A two IRF Members BFD MAD configuration can be found here (as example) BUT, for sure, the IRF Configuration Guide for your Comware based Switch Series (whatever it is) has some BDF MAD deployment examples too.

[Stefano Colombo]

Hello 

I'm setting up the same configuration, an IRF stack with 4 unit, but we have 2 unit in a data center and other 2 in another room.

Regarding the BFD connection using an assisting device, it would be possible to have 2 assisting devices , one per room, connected together, but not as a stack ?

thanks

[parnassus]

Hi @Stefano Colombo  well let me say that if you already stretched a single IRF domain across two rooms (two IRF members on Room A and the other two IRF members on Room B), using a single BFD forwarding device (on any of the rooms) shouldn't be an issue in terms of cabling effort...documentation reports no particular restrictions in terms of number of BFD assisting switches (I mean there isn't anything in the form of "Two forwarding MAD devices can't be used" statement) but since BFD requires no STP on involved ports....how then you will deal with the fact that you want to use two (one per room) BFD forwarding switch just simply interlinked together (and so not part of an another dedicated IRF domain)? Indeed it is suggested the possibility to use another IRF (with a different Domain ID with respect the "main" IRF you want to protect against split brain scenarios) as the BFD forwarding entity so you can geographycally split it too (placing one fragment on Room A and the other in Room B) and continue to stay with STP not enabled on BFD involved ports. It looks like a sort of looping idea but...consider not implementing it that way will led you to two BFD Switches uplinked together and connected, each one, on each side of your stretched IRF...those two BFD Switches will form a network loop because the IRF acts - stretched or not it will be - as a single logical identity but the two BFD Switches instead act as two separate logical entities connected to your IRF and the loop will form. Am I wrong?

Not specifically related to BFD MAD scenarios...but this old blog post is really interesting about stretched IRF scenarios (Pro/Cons).

[Stefano Colombo]

HI @parnassus 

We're still in pre-deployment phase, so nothing configured yet.

The scenario is as such.

An IRF domain with 4 5710 devices, 2 per room,.

On each room there are Cisco 9200 swittch wich are connected with normal port channel.

Those 9200 switch, one per side, are the one we can use to configure MAD.
So excluding LACP mad I thougth ad BFD MAD, but your point about STP is correct.

I Thought that requirement for STP disabled on BFD MAD port was only referred on the physical port on the 5710 not on the link between two supporting devices.

Using L3 wouldn't be ok having a dedicated VLAN spanned across the Cisco 9200 switches used for BFD ?

If neither LACP nor BFD could be used in this scenario what can we use then ?

[parnassus]

Ciao @Stefano Colombo 

I see the limitation comes from having the two Cisco switches working respectively as standalone units. Since you need STP enabled at some point (I believe you can't avoid that in the IRF 1 East side - 9200 East side - 9200 West side - IRF 1 West side loop), exactly the STP will kick in to cut the loop.

"I Thought that requirement for STP disabled on BFD MAD port was only referred on the physical port on the 5710 not on the link between two supporting devices."

But STP disabled on ports facing the sides of the HPE 5710 IRF (ports that are on each Cisco 9200) will not save you when both Cisco 9200 will be linked together to transport BFD VLAN messages (and Layer 3 is not an option since you need to transport the BFD VLAN at Layer 2 and, also, if going into Layer 3 between both Cisco 9200 would be an option, then you should create two separated Spanning Tree topologies - one per Cisco 9200 - to avoid STP to kick in among the two...something like using STP BPDU filtering on Cisco 9200 - Cisco 9200 inter-link(s)...I mean each Cisco 9200 should, in that case, have its own Spanning Tree...in any case Layer 3 is not an option with BFD MAD).

The only ways I can think of to overcome this cul-de-sac is:

1. To use another dedicated IRF/Virtual Stack (ma temo la terapia sia peggiore della malattia che si intende tenere sotto controllo).
2. To try to link directly IRF Members on both rooms to one single BFD assisting switch (not necessarily one of the two Cisco 9200) installed into Room A or into Room B (or, possibly, in another different room)
3. Split the IRF stretched domain into two IRF Domains (2 x 5710 will form an IRF Domain on Room A and 2 x 5710 will form another IRF Domain on Room B), at that point implementing any MAD will not be an issue (within a Room). Both IRF Domains will be then interlinked by means of a proper LACP LAG.

With regard to scenario 3 there are for sure other requirements that should be considered too (where routing will take place if you're dealing with two IRFs and not with just one single stretched IRF, as example). There are also technologies as DCI/EVI/VXLAN that will help in interconnecting two Datacenters at Layer 2 level...but probably these weren't considered since you're just planning to "split" a single IRF into two separated DC Rooms...not interconnecting two geographically separated DCs.

Hope not to be wrong...

[Stefano Colombo]

Ciao @parnassus
I did not mention that there’s another expected connection between the 5710 IRF stack and a stack of 2 Cisco 4500
That connection will be a 4x 10Gb aggregated link, one for rack 5710, to the two Cisco 4500
In stack
Can we use that same link for MAD ? LACP MAD ?
Thanks

[parnassus]

Ciao, if the Stack of Cisco 4500 - acting as the intermediate MAD assist "device"- supports extended LACP for MAD...it could work.

At that point on the Cisco 4500 Stack side you only need a "normal" BAGG (Bridge Aggregation) with LACP and on the HPE FlexNetwork 5710 IRF side you just need to setup a MAD LACP as per documentation.

Having a Cisco 4500 Stack available would probably solve your issue but it all boils down to be sure it supports extended LACP for MAD.

Out of curiosity: are the peer systems located in Room A consuming IRF services/features "dual homed" to both IRF Members on Room A and IRF Members on Room B (and vice-versa for peer systems on Room B)?

[Stefano Colombo]

Hello @parnassus
How or where can we check whether the 4500 supports the “extended” LACP required?
What extension are they referring to ?

The peer system, at the moment, are not thought to be multihomed but it might be a future option
I understand your point, however, and it’s being evaluated to fail back to a two IRF design
Thanks

[parnassus]

---

@Stefano Colombo wrote: How or where can we check whether the 4500 supports the “extended” LACP required?
What extension are they referring to ?

---

I stole the word from the LACP-MAD Overview: "LACP-MAD is implemented by sending extended LACP Data Units (LACPDUs) with a type length value (TLV) that conveys the Active ID of an IRF virtual device. The Active ID is identical to the IRF Member ID of the Master and is thus unique to the IRF virtual device. When LACP MAD detection is enabled, the IRF Members exchange their Active IDs by sending extended LACPDUs." (taken from an HP ProVision OS based web page about LACP-MAD! that's because some HP ProVision/ArubaOS-Switch switch series can be used as MAD assist devices...but those exact same words came from this H3C Guide...so perfectly Comware OS based switch series related).

The point is, if I understood you correctly, that you are dealing with a Stack (VSS) of Cisco 4500...and so....I believe you are forced to activate (Cisco 4500 stack side) a MEC (Multi-Chassis EtherChannel) against the IRF (stretched or not the IRF is, stretched or not the Cisco 4500 VSS is)...so the question is not only if a Cisco 4500 VSS supports to act as a MAD assist device from the IRF Stack standpoint but also if the MEC (LACP) you're going to use to interconnect the VSS Stack with the IRF Stack supports extended LACPDUs (and, if so, it would ALSO let the IRF to act as an IRF MAD for the VSS...not that will be necessary but just to prove that a LACP could transport required extented LACPDUs either way we "source" them)...and I fear that - from what I've understood (I'm not a Cisco expert) - this is probably not supported (see here)...I mean - have a look at pages 15 - 20 - it seems to me that "standard" MEC LACP - also referenced as "cross-stack EtherChannel" - can't be used for MAD in the "Cisco way" and for a "pure Cisco scenario" standpoints...so, based to requirements/restrictions noted, exploring how Cisco solves the MAD deployment methods from the Cisco standoint (to infere if methods are reversible and so applicable to IRF as "orginator"), it seems to me you need to use Cisco ePAgP (which requires Cisco PAgP) on the VSS which is an alternative way to deploy MEC to upstream/downstream devices with respect to MEC LACP (keep in mind that HPE IRF doesn't support Cisco PAgP) and it also suggest me that the usage of ePAgP to deploy the MAD mechanism is the only way when upstream/downstream devices are connected with a MEC (as per our case)...if Cisco doesn't rely on MEC LACP to deploty its MAD mechanism (indeed preferring the ePAgP on supported involved devices) but forces you to fail back to the alternative Dual Active IP BFD mechanism it means that "standard" MEC LACP are not so developed in terms of extended LACPDUs (otherwise a MEC LACP would be enough)...that's the reason I believe could be an hard way relying on a VSS of two Cisco 4500 for MAD...much better a basic Aruba switch a that point.

[Stefano Colombo]

Hello @parnassus

could you advise for an Aruba switch we could use as supporting device for MAD , in order to solve the issue ?

We could install it on one side and connect all the IRF nodes and use BFD MAD 

thanks

SC

[parnassus]

Hi Stefano, as far as I know below is a (partial?) list of ArubaOS-Switch operating system based Switch Series currently supporting LACP MAD Passthrough:

1. Aruba 2530 (also known as HP 2530)
2. Aruba 2540
3. Aruba 2920 (also knonw as HP 2920, discontinued in favor of Aruba 2930F/2930M Switch series)
4. Aruba 2930F / Aruba 2930M
5. Aruba 3810M
6. Aruba 5400R zl2

On their latest Management and Configuration Guides you should be able to find a small dedicated LACP-MAD chapter (OTOH also this document could be useful in case of troubleshooting).

All the above Switch sereis, excluding the Aruba 2920, supports ArubaOS-Switch up to latest 16.10.0013 (ArubaOS-Switch for Aruba 2920 seems to have stopped at 16.10.0012), I suggest you to work with latest version whenever possible.

Giving the above list a small 8 ports HP/Aruba 2530 Switch should suffice.

[Stefano Colombo]

Hello @parnassus 

Thanks a lot for your help