Help with IRF on pair of 5700-48G-4XG-2QSFP+ / JG894A

[Raland]

Have a pair of 5700s serving as the core for shiny new three node HC 380 install. I'm pretty versed on the 2920 series. I'm pretty confortable working in the comware universe now but having some issues figuring out the 'best' way to setup IRF.

Network Image
I'm still a little confused on how to cable it. The switches came with a 40Gb to four 10Gb splitter cable, which seemed like the perfect thing to use in my case. FGbe 53 became 1/0/53:1-4
1/0/53:1-2-3 to the hosts
1/0/53:4 to the other 5700 (2/0/52)

2/0/53:1-2-3 to the hosts
2/0/53:4 to the other 5700 (1/0/52)

So I did the basics on switch 2:
irf member 1 renumber 2
irf member 2 priority 30

When I try and bring the interfaces back up (2/0/53:4 and 2/0/49) I get this error.
Bind all interfaces in the same group to IRF/PEX ports or cancel the bindings on all of them.

So I tried native 10G ports 2/0/51 and 2/0/52 with the same error.

I found in another article here that explains this error, if you use any port in IRF you have to use them all. So I'm going to lose ports 49/50 Correct? (other than IRF to aditional switches)

Most examples I have seen use both the 40G ports. I just got my first 40G ports. I'm geedy and don't want to toss those off to stacking duties if I don't have to.  Should I keep my IRF on the 10G and lose the two ports or use both 40G?

My my network traffic for the vms is not very high, migrating off the 2920s with two 1G ports to the old hosts. With the ability to use the splitter cable, I have more available ports (2x40G with 2 splitters = eight 10Gbe). Thanks!

[Vince-Whirlwind]

Logically, your bandwidth allocation priority needs to be Core stacking first, blade enclosures/end of row datacentre switches second.

In some networks (scientific/modelling type applications) there may be a high speed datalink that takes priority over local servers, but you should never be allocating higher bandwidth to uplinks than to the internal core.

If I had 5700s for a Core, then with only 2x QSFP ports in each, I would consider those ports for IRF only. If I had a couple of blade enclosures that wanted 40Gb uplinks, I would add a pair of 5900s to the stack, or just use 5900s in the first place, depending on how much port capacity I needed in the Core.

But if I wanted to cater for 40Gb connectivity more generally, I would probably go with the 7050 or 7280 in the first place.

[Raland]

Thanks for the input. After some further thought and some discovers I think I'm going to do as you suggested and IRF the two 40G ports.
While keeping the 40G out of IRF and using a splitter cable would give me access to more ports it wouldn't give me any SFP+ ports. Right now I'm using 10G fiber modules to connect the 5700s to my production network so I can't use IRF on the 10G ports at the moment.

The JD097C DAC cable does not work between the 2920 and the 5700. I didn't actually buy it to do that connection so the correct cable should fix this just need to research the correct one.   

[parnassus]

Just theoretical speculations:

1. Since 5700 supports IRF Ring Topology even with only two units (which, to me, remains a counter intuitive concept) one would ask if instead of using (see point 2) the QSFP+ (40 Gbps) port on each IRF Member (IRF Port 1 of IRF Member 1 to IRF Port 2 of IRF Member 2) to create the normal two IRF Members Daisy Chain topology, what would happen (is it good or not?) if splitting the QSFP+ port into its 4 TenGigabit (breakout) ports and then bound a pair of them (10+10 Gbps) to IRF Port 1 on IRF Member 1 and bound the other remainig pair (10+10 Gbps) to the IRF Port 2 on IRF Member 1, doing the same on the other 5700 and then create the IRF Ring Topology (IRF 1/1 to IRF 2/2, IRF 1/2 to IRF 2/1)...it would make sense?
2. Just use QSFP+ 40 Gbps as a single IRF physical link (as you planned) and do simple IRF Daisy Chain between both 5700.

Probably (1) is a total practical non sense since yes, it's Ring but it behaves like Chain and, what's worst, you need to breakout QSFP+ (more cables)...just speculation.

Better a single 40Gbps IRF link or a pair, 20 Gbps each one?

Speculation apart, HPE FlexFabric 5700 Switch Series IRF Configuration Guide reports these QSFP+ restrictions:

QSFP+ port restrictions

You can use a QSFP+ port as an IRF physical interface or use the breakout SFP+ ports of the QSFP+ port as IRF physical interfaces. To use the breakout SFP+ ports, you must use the using tengige command to split the QSFP+ port into four 10-GE breakout interfaces. When you use the breakout interfaces of a 40-GE port for IRF links, follow these restrictions and guidelines:

• You must use all or none of the four 10-GE breakout interfaces as IRF physical interfaces. The
  four breakout interfaces can be bound to different IRF ports. <-- here you will be basically OK.
  
• Before you bind one 10-GE breakout interface to an IRF port or remove it from the IRF port, you
  must shut down all the other 10-GE breakout interfaces. If any of the breakout interfaces are in
  up state, the bind or remove action will fail. <-- this could be a problem if you need to manage a breakout interface singularly.
• Bring up the breakout interfaces after you complete the operation. <-- OK, that's reasonable.

[Raland]

That is an interesting idea but this would use up all the 10G and both 40G interfaces for IRF leaving me with only 1G (or mutiple 1G) connections to the 2920 production network switch. Even if you split a 40G you can't just use the four 10G port for IRF. In fact I may have tried this and it gave me an error that all the IRF ports in a group need to be the same type?

This is starting to seem overly compliacated. It shouldn't take days and $1000 worth of cables to stack two switches. I have a good hour return to service window. Easily enough time to grab a spare switch off the shelf and replace a failed unit. Do i even need IRF?

[parnassus]

I was just speculating on the idea of one IRF Port (to deploy a Daisy Chain topology) bounds to a single 40Gbps physical link (QSFP+) versus two IRF Ports (to deploy a "fake" Ring topology) bound to, each one, a pair of breakout 10Gbps ports (the QSFP+ splitted into four 10G).

I already know my idea is unpractical and, probably, not so wise/smart.

I think QSFP+ IRF Port restrictions sentences say it clear: you should be able to do that by splitting the QSFP+ (without using other 10Gb ports your 5700 has).

[Raland]

This seems to be what you are describing? Can you explain why I would need eight 10 GBbe interfaces for this?

[parnassus]

Sorry, I underrated the value of saying that a QSFP+ port can be splitted into 4 SFP+ without saying that, on the other end, you can so use just 4 SFP+ instead of another QSFP+ port (so saving it for other usage). Yeah, that's totally asymmetrical (physically speaking: because on the first IRF Member you will use the XGE53:1-4, on the other one you will use 10G SFP+ Ports 49, 50, 51 and 52)

XGE53:1-2 binded to IRF Port 1 (on IRF Member 1)

XGE53:3-4 binded to IRF Port 2 (on IRF Member 1)

49 and 50 binded to IRF Port 1 (on IRF Member 2)

51 and 52 binded to IRF Port 2 (on IRF Member 2)

IRF Port 1 (on IRF Member 1) linked to IRF Port 2 (on IRF Member 2) and IRF Port 1 (on IRF Member 2) linked to IRF Port 2 (on IRF Member 1).

Since you own the QSFP+ to 4xSFP+ spitting cable yet it should be possible do that.

That was my speculation (sorry if I confused things a little bit more than necessary).

Clearly the same can be achieved by definitely not using any QSFP+ interface but just using Ports 49, 50, 51 and 52 on both IRF Members (aggregated together and all four binded to a single IRF Port 1 on IRF Member 1 and on IRF Port 2 on IRF Member 2 as per 2 IRF Members Daisy Chain topologu *or* used in pairs to create that 2 IRF Members Ring topology discussed above).

In any case (QSFP+ and SFP+ or SFP+ and SFP+) you will need at worst 4 10G ports per IRF Member, not 8. Sorry again.

[Vince-Whirlwind]

"...4 10G ports per IRF Member, not 8." Per link you mean?

[parnassus]

No, in my speculation I meant per IRF Member.

I thought: of four 10G interfaces used (as breakout of QSFP+ or as native SFP+), a pair bound to IRF Port 1 and the other pair bound to IRF-2 Port...so each IRF Link (1 to 2 on IRF Member 1 and 2 to 1 on IRF Member 2) is 10G+10G...the grand total is always 40G.

Just speculation.