BladeSystem - General
cancel
Showing results for 
Search instead for 
Did you mean: 

Use of single interconnect module - possible?

 
SOLVED
Go to solution
Thor T
Occasional Visitor

Use of single interconnect module - possible?

Hi guys,

We've got a c3000 system with two GbE2c Ethernet Blade Switches and two 3Gb SAS Switches. The enclosure is fully populated...

Each blade installed has two mezzanine cards installed - one dual port NIC and a P700 Smart Array giving the blades access to an MSA2200.

I have a single 16 port FC switch and three FC HBAs that i want to install in the blades and the enclosure.

I've read all the documentation provided but can't confirm whether it is possible to run a single interconnect module (FC switch) and have it connect to the HBAs in the blades.

The ethernet blade switches run in a dual start topology and i can confirm that they are both being used to full capacity at times. This is why i won't replace any one of them.

The SAS switches however run in a different mode were one is passive. I tried pulling one out and replacing it with the FC switch but the insight manager started complaining about incompatibility and missing links.

I can see that the FC switch once installed has connectivity to the HBA installed in one of the blades but the SAS connectivity won't function if i keep this setup.

What do i do? How can i have the single FC switch and three HBAs installed and run without configuration errors - if at all possible?

FYI, blades are running ESX servers.

Best regards,
Thor
6 REPLIES
Matti_Kurkela
Honored Contributor
Solution

Re: Use of single interconnect module - possible?

The picture you attached indicates you're using half-height blades.

As I've understood, a half-height blade has only two mezzanine card slots. By your description, both slots are already filled in all blades of your set-up.

The blade architecture is designed to work best when each blade in the chassis has a similar I/O configuration. You can have a blade with a mezzanine slot corresponding to one (or two) of your I/O modules left empty, or you can have (some of) your mezzanine card's ports connected to an empty I/O module slot. What you cannot have is a mezzanine card and an I/O module of incompatible types connected to each other: if you try to do this, the OA will hold the appropriate components in a powered-down state.

You did not specify your blade configuration explicitly, but apparently only one configuration is possible: your blades have the dual-port NICs in mezzanine slot #1, and the P700 SmartArray in mezzanine slot #2.

Replacing the mezzanine #1 with the HBA would require removing the NIC mezzanine cards from mezz #1 of all blades, and installing the FC switch module in the I/O slot 2.

Because mezzanine #2 is hardwired to both I/O module slots 3 and 4, installing the FC HBA to mezzanine #2 would require removing the SmartArray cards from each blade, and the corresponding SAS switches.

You could then put the FC switch module to either slot 3 or 4 and you'd get one FC port from each blade that has the FC HBA installed in mezzanine #2. The other port in each dual-port HBA would be wasted until you could get a second FC I/O module.

Any other combinations would lead to components not powering up.

MK
MK
Sivaramakrishna_1
Honored Contributor

Re: Use of single interconnect module - possible?

As u have a single FC passthrough module, recommaded action for this would be removing Mezz card from Slo1 and remove interconnect 2 and populate with FC mezz and FC passthrough.
In this case it will enable the both fc modules on slot 2.

If u wish to remove SAS controller then u need to remove both 3 and 4 interconnect and populate mezz in 2 and passthrough in either 3 or 4, but this will enable access to a single port on the card
Thor T
Occasional Visitor

Re: Use of single interconnect module - possible?

Hi guys,

Thank you both for the detailed explanation and suggestions.

I now fully understand what my options are. I do, however, require some more suggestions or recommendations.

On the back of the chassis there's a small half size module that looks like a 4-port switch. What is it, why is it not selectable in the OA webui? I've read in the documentation that each blade has a built in NIC, and i can confirm it by seing that each blade has two NICs through Server NIC Information in OA webui.

Does the little 4-port module act like a switch to all the installed blades and their onboard NICs?

How can i tell if that switch is active or maybe passive due to the installed mezzanine NICs and GbE2c switches?

Would it be a bad idea if i follow the first suggestion which was to remove all mezz #1 from the blades, replace with FC HBAs and install FC SW in slot # 2? Would that not still provide networking access to the blades from that little 4-port module? (see attached image in 1st post).

-

In our VMware setup each host has vmnics that are -i presume- the interfaces to each dual-port NIC mezzanines installed in the blades. If my thoughts above are true and i remove the mezzanines then i'd still have access to the network through the onboard NICs in the blades, no?

Your input is very appreciated.
Sivaramakrishna_1
Honored Contributor

Re: Use of single interconnect module - possible?

Hi,

That 4 port (NIC) is not Data traffic. That ports will be used for managing the Enclosure, The ports are (from Left to Right) 1. Enclosure uplink and service port 2. Enclosure Downlink port 3. iLO Onboard Administrator port 4. Onboard Administrator port2(reserved for future use)

So for accessing the Console of the servers you need to use the port 3. You can set the IP from the OCP panel available from front.

These are 100Mbps ports and dedicated for Enclosure management and accessing the consoles of the servers installed in Enclosure only.

If you want to connect to FC Storage the only option is available for you to remove the Mezz card 1 and install with FC Mezz and install FC Passthrough in inter connect 2.

But you will be having only to 2 NIC ports (onboard ports of each server) with Interconnect switch1 only.

Is there any SAS storage is going to connect to this enclousre?. If not then remove the Mezz 2 and Interconnect 3 and 4 so that you can use additional NIC ports from Interconnect 2 and FC through either 3 or 4.
Matti_Kurkela
Honored Contributor

Re: Use of single interconnect module - possible?

Each half-height blade has 2 built-in NICs, which are connected to I/O slot #1.
In addition, you currently have dual-port NICs in Mezz 1, which connects to I/O slot #2.

Replacing the dual-port NICs in Mezz 1 with FC cards and the I/O module #2 with the FC switch would give you FC capability, but would halve your network bandwidth: you would still have network connectivity through the integrated NICs and the I/O module #1.

If you can accept the reduced network bandwidth, it could be a valid solution to your problem.

In a VMware environment, it's impossible to say what your vmnics might be without knowing more about your VMware vswitch configuration.

Sivaramakrishna's explanation about the small 4-port module is correct.

MK
MK
Thor T
Occasional Visitor

Re: Use of single interconnect module - possible?

Thanks guys.
Greatly appreciated!

Best regards,
Thor