Need advice on c-Class blade networking configuration

[Banibrata Dutta]

Hi,

Consider this a question from a Blades n00b.

I'm trying to figure out the best way to mirror a Proliant (rackmouts) based largish solution (about 20 servers), onto c-Class blades. My question today, is more about the networking aspects in c-Class.

All these servers need to communicate with each other, and in "Proliant rack-mount" based solutions of similar dimensions, we generally have 2 sets of Gigabit Ethernet conn. towards a pair of redundant switches (which are interconnected) from each server for this purpose. Also, we have 2 sets of Gigabit Ethernet conn. towards the external world -- again thru a pair of redundant (& interconnected) switches.

One additional (important) requirement is that the external connectivity must have a "fan-out" of 64 ports of 10/100Mbps.

The question is, how do I mirror this solution on c-Class blade architecture, without compromising on redundancy (s.a. duplicate LANs for both internal & external usage), and yet have that 64 ports of fanout for external Ethernet connectivity ?

My current understanding is that I'd need:
- 2 units of 16-port Gigabit switches per chasis, for internal connectivity, that would be interconnected. (10 ports each for servers, and 2 each for switch interconnects).
- X? units for a total of 20 ports (redundant 10) of Gigabit Ethernet Pass-Thru modules, per Chasis (all of which terminate on "external", standard ProCurve 24-port switches (eg. 2480s), which have auto-sensing/auto-negotiation s.t. they'd then be used for 10/100Mbps fanout.
- And, how do I interconnect the enclosures ?

My main confusion is around what all to put on inbuilt Switches, add-on Switches, external Switches, what all needs to go thru Ethernet pass-thru.

Appreciate any guidance in this regard.

Thanks & regards,
Banibrata

[Raghuarch]

I will suggest you to Go with the 2 Virtual Connects Ethernet modules Per-Chasis.

Please go through the below Docs for Further info.
http://h71028.www7.hp.com/ERC/downloads/4AA0-5821ENW.pdf
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c00814156/c00814156.pdf
http://h20000.www2.hp.com/bc/docs/support/SupportManual/c01386629/c01386629.pdf

how do I interconnect the enclosures ?
There is a Enclosure Link Option Available in OA Through which you can Connect the Enclosure together.

[Banibrata Dutta]

Thanks for the links and suggestion about including 2 Ethernet VC modules/chasis. I am yet to go though them, so I might add few more question on this thread.

Regarding Enclosure interconnect via the OA's ethernet ports isn't the kind of interconnect I was thinking of. Indeed it's a useful feature I'd include in the solution. To re-phrase my question, I was thinking as to how, I could provide interconnectivity between the services accross 2 chasis, while ensuring that there's enough bandwidth, and enough redundancy ?

I'll read those docs, they might already have answers for that question.

Thanks & regards,
Banibrata

[Lmm_1]

Hello Banibrata,
Im not sure if im getting your question, but anyways... here are my toughts

You can make OA stacking, there are 2 dedicated ports for that on each OA, basically it will be only for management porpuses... if you use blade switches there will be 1 internal dedicated port connected to the OA, then you will have 1 single solution for enclosures and interconnects management.

Regarding ethernet connectivity, you can use blade switches, most common configurations that I´ve seen are 2 blade swithes (Cisco 3020, BNT GbE2 or VC), those will get all the embeedde NICs on their 16 downlinks, then you can use 2 sets of etherchannels to get all your vlans communicating to the external switch. Depending of the Blade, Switch and cabling configuration you will get a fully redundant solution.

Sometime important to mention, Cclass enclosures will work only 1000 full duplex.

Rgds,
lmm

[Banibrata Dutta]

Thank you Lmm.

You do mention an important point about c-class being predominantly 1Gbps full-duplex only, however, the moment we connect it to external Auto-sensing, Auto-negotiating L2 Switches s.a. the Procurve 2480 etc., we are able to interconnect to any 10/100Mbps device as well, right ?

I've read the VC document. Based on my understanding after the quick initial read, I believe it acts as a L2-switch & a patch-panel, but instead of hard-wired copper connection, the patch-panel's cord-patching can be changed thru the VC mgt GUI. Is that a correct (if rough) understanding ?

Although this might hv been already answered above (in raghuarch or lmm's mails), but I guess I'm too n00b to notice it... what are the best practices for carrying bulk amounts of application network traffic (& not just chasis/shelf management -- as in OA traffic), between multiple chasis. Say for instance, I want about 4Gbps of full-duplex bandwidth to be available between 3 set of Enclosures/chasis, how do I manage it ?

thanks for the answers.

regards,
Banibrata

[Banibrata Dutta]

To refine my following question a bit more:

... what are the best practices for carrying bulk amounts of application network traffic (& not just chasis/shelf management -- as in OA traffic), between multiple chasis. Say for instance, I want about 4Gbps of full-duplex bandwidth to be available between 3 set of Enclosures/chasis, how do I manage it ?


Would the embedded L2-switching functionality of VC, be available to also interconnect 2 different chassis/enclosures, in any way ? Or, I must use a Switch-blade ? Effectively, how do I interconnect the VC's in 2 different enclosures ?

[Lmm_1]

The VC will be an expensive solution, with those you are not able to create a VC Domain with multiple enclosures, not yet at least. I think that feature will be included on next VC Firmware release.

You can accomplish your goal using blade Switches... it will depend only of the configuration, take a look on the attached document, it has some examples of Gbe2c Blade switches and Cisco Core Switch.

Rgds,

[rick jones]

The (non 10gig) NICs in the blades themselves are "always" 1000 Mbit/s full duplex. If you use pass-thru modules then the external connections will also be 1000 Mbit/s full duplex. When using switches in the chassis, the connection from the blade to the chassis switch will be 1000 Mbit/s full duplex. I _believe_ that the external ports on the chassis switches will negotiate to whatever the peer will do. However, I've not even tried to run anything down at 10/100 for years...

Can you expand a bit on why the external connectivity "must" be 64 ports of 10/100?

[Banibrata Dutta]

Hi Rick:

The solution is a Telecom-solution that must inter-operate with the Telecom-operators' existing network infrastructure, which is all 10/100Mbps as of date. That's where the 10/100Mbps port requirement comes from. The 64 is based the the effective throughput required out of and into the solution from other network-nodes/entities in the Operator's network -- ~6.4Gbps effective, full-duplex throughput.

Thanks for the information.

cheers,
Banibrata

[rick jones · Accepted solution]

I'll probably miss some of the requirements, but going with some of the 1/10G VC Ethernet modules, using the 10G uplinks to some external ProCurve switches sounds like one way to go. You don't have to use the "VC" part of a "VC module" if you don't need/want to.

That way you would have either two or four 10G links to the external switches and then your 64 10/100's from there.

For interconnecting the enclosures you could go with an all-10G external ProCurve switch (iirc the 6400 is such a switch) and then connect from that to 10/100/1000 switches with 10G links. (iirc that could be the 3400 series or perhaps others).