Question re. using 5700 as ToR with 5406r distribution

[emsc]

Hi,

We have in our rack 2x5700 (JG894A) that will be used as ToR.

The idea is to IRF both of them and uplink to our two core 5406r.

The initial thought was to use:

2xSFP+ (from each switch) for IRF and 2xSFP+ (from each link) for uplink to our distribution. This doesn't seem to work because it seems like I need to add all 4 SFP+ ports to IRF. I can't use 2 for IRF and 2 for uplink.

Questions:

• Is that correct? Or am I doing something wrong in the config?
• Can I use the 2 QSFP+ ports from each switch for IRF, and 2x SFP+ as uplink?

Thanks!

em

[parnassus]

AFAIK (check IRF requirements and restrictions, especially the paragraph named "IRF physical port restrictions and binding requirements" for your specific Switch series on its IRF Configuration Guide manual)...SFP+ ports are grouped [*] in groups made of 4 physical ports each...so if you just used two SFP+ ports of that port group for creating a logical IRF port then the others two remaining SFP+ ports can't be used for any other function other than being part of an existing (or a new) logical IRF port on the considered Switch.

[*] better worded here:

"SFP+ ports on the HP 5900AF-48XG-4QSFP+ Switch, HP 5900AF-48XG-4QSFP+ TAA Switch, HP5920AF-24XG Switch, and HP 5920AF-24XG TAA Switch are grouped by port number in order, starting from the one numbered 1. Each group contains four ports. Ports in the same group must be used for the same purpose. If you use one port in a group as an IRF physical port, the other ports in the group cannot be used for any other purposes."

[emsc]

Does IRF refers only for stacking? 

Since I can't split the 4 SFP+, I would like to use the QSFP+ for stacking (directly connected between both switches) and the SFP+ for uplink to our 5406r

Is this possible?

[parnassus]

Yes, it should be possible...but you have to plan a reconfiguration of your IRF Stack (with its IRF logical ports) to use the one 40 GbE QSFP+ port (or, if you can afford, at maximum two binded to the same IRF logical port for resiliency and load balancing) instead of using, as happens now, 2 x 10GbE SFP+ ports...doing so you will free your actual four SFP+ ports (four in total, per Switch) and then you will be finally free to use them to create your LAG(s), each LAG, terminating on each respective Aruba 5400R zl2 Switch...or use those ports at your convenience.

On the HPE FlexFabric 5700 Switch Series, the IRF physical interfaces admitted are:

• 10 GBase-T ports (group restrictions as described above)
• SFP+ ports (group restrictions as described above)
• QSFP+ ports (admitted usage: a single 40GbE QSFP+ port as an IRF physical interface OR breakout [*] SFP+ ports of the QSFP+ port as IRF physical interfaces)

[*] when you follow the breakout SFP+ ports rway (so the the single 40G QSFP+ port splitted in four 10G SFP+ links) then you fall again on the group restriction case (basically, it's "all or none").

For details see the HPE FlexFabric 5700 Switch Series IRF Configuration Guide (valid for 2422P01 and later versions), Hardware Compatibility paragraph of the Setting up an IRF fabric first chapter.

[emsc]

Thanks for the answer!

This is a vanilla switch, so reconfiguration is not a problem :), and the idea is to use both QSFP+ ports (on each switch) for the IRF)

So:

• Both QSFP+ on each switch will be part of IRF 1/1
• SFP+ will be used for uplink to 5406. 

Question:

You mention:

doing so you will free your actual four SFP+ ports (four in total, per Switch) and then you will be finally free to use them to create your LAG(s), each LAG, terminating on each respective Aruba 5400R zl2 Switch

Do I need to use all 4 SFP+ (per switch) for uplink or can I use only two, and maybe later add the other two?

[parnassus]

Absolutely not, LAG/Port Trunking leaves you free to choose two (or more...up to eight? I don't remember!) ports to be aggregated together to form the BAGG...I wrote "4 SFP+" but was just a way to say you're definitely free to do what you want with those 4 SFP+ ports.

The point, OTOH, is that if the LAG you create is originating on ports of the IRF Stack (two cases: all ports on one IRF member - let's say 2 SFP+ on IRF member 1 - *OR* all ports distributed across both IRF members - let's say 1 SFP+ of IRF member 1 and 1 SFP+ of IRF member 2 - which is totally possible...not always convenient...due to the virtual switching nature of the IRF Stack)...you have then to terminate them, whichever case you follow, on one and only one Aruba 5400R zl2 at Arubas side at time...at least until - and you didn't mentioned if you are planning to deploy it or not - you've a VSF stack (another virtual switch) waiting on the LAGs terminating side...in absence of a VSF on the Aruba side you're forced - that's *the viceversa* - to terminate LAGs originating on the IRF Stack on each Aruba 5400R zl2 respectively (and so separately)...and, probably, that is what you originally wanted to do.

---

emsc wrote: ...and the idea is to use both QSFP+ ports (on each switch) for the IRF

That's good! IRF daisy chain topology, one IRF port per IRF member...multiple physical ports per IRF port...plenty of bandwidth (enhanced throughput - 40+40 Gbps - between IRF members) and good resiliency (two 40GbE QSPF+ links are better than one)...you're lucky!

[emsc]

We have two 5406 which will be configured with VSF.

So, the connections will go:

switch5700A.sfp1 -> switch5406A.sfp1

switch5700A.sfp2 -> switch5406B.sfp1

switch5700B.sfp1 -> switch5406A.sfp2

switch5700B.sfp2 -> switch5406B.sfp2

Makes sense?

[parnassus]

It does (it's a mesh, you're "crossing" respectively SFP+ ports 2 and 1 of IRF Member A and B with respectively SFP+ ports 1 and 2 of VSF Member B and A)...clearly we can say nothing about the real (planned) traffic flows among all these units (because we don't know nothing about it...but IRF is engineered IMHO to manage it intelligently enough)...but in the end...deployed *that* way you should have full resiliency against a node/link failure and so all edge devices connected are reacheable by (and reach) all other remaining edge devices.

Isn't that your ultimate goal?

I guess that also a schema like this (not meshed) should be potentially OK...but, as always, all depends about your requirements (I mean: mesh or not mest...if you just lose an entire node for a KO - it is "A" or it is "B" - whichever side will suffer the node lost - IRF on VSF - the meshed topology setup, exactly as the non meshed one, will not help you to continue to sustain all the edge devices lost with the entire node failure):

• SW 5700 IRF node A SFP+ port 1 irfLAG 1 <--> vsfLAG 1 SFP+ port 1 SW 5406R zl2 VSF node A
• SW 5700 IRF node A SFP+ port 2 irfLAG 1 <--> vsfLAG 1 SFP+ port 2 SW 5406R zl2 VSF node A
• SW 5700 IRF node B SFP+ port 1 irfLAG 2 <--> vsfLAG 2 SFP+ port 1 SW 5406R zl2 VSF node B
• SW 5700 IRF node B SFP+ port 2 irfLAG 2 <--> vsfLAG 2 SFP+ port 2 SW 5406R zl2 VSF node B

[emsc]

Thanks again!

I'll give it a try in the following days and update the post.