Recommended VMWare setup for C3000

[UselessUser]

Hi,

I have a C3000 with BL465c blades.

I have 2 VC-FC modules and 2 HP ethernet modules.

The 2 HP ethernet modules are about to be replaced with 2 VC Ethernet modules (What does this give us extra?)

We are installing ESX 3.5 on the blades (On local mirrored 72GB disks) and I am wondering basically about networking and FC connectivity...

I have the Emulux HBA mezzanine cards but do not have any ethernet mezzanine cards, so each ESX host will only have 2 NIC's I believe?

Is this going to be a problem (If so what can I get to relieve it etc).. I kind of thought in terms of throughput etc only having 2 NIC's is not a problem as it is the external ports (The VC-Ethernet modules) that are going to control this really??

Any tips or guides would be appreciated?

[Rob_69_1]

Hi, I have a similar setup, but yet I don't have switch redundandcy. Anyway.

- having just 1 FC mezzanine with 2 ports is correct, because if you put it in the right slot (can't remember which one, but it's easy to find in the docs) then you'll have 1 HBA port pointing to the first FC switch, and the second one pointing to the other one. So if your multipath drivers are working ok, you'll be fine with your SAN etc.
- The embedded NICs: if you want redundancy, you need another mezzanine NIC card with 2 ports on EACH blade. In fact, the "embedded" ports are natively BOTH mapping to just ONE ethernet switch. I think they did so because if you want to use bonding, you can do it easily... I don't really know. By the way, if you put another card on the mezzanine, those 2 ports would map to the OTHER switch, so you're up and running with ethernet redundancy too :-)

Cheers

[Steven Clementi]

"I have the Emulux HBA mezzanine cards but do not have any ethernet mezzanine cards, so each ESX host will only have 2 NIC's I believe?"

- Each server has 2 embedded NIC's. They are connected to the BAY1 Interconnect.

- The HBA's should be placed in Mezz slot 2 so that the ports map out to Interconnect BAY3 and BAY4.


The typical ESX configuration usually requires a minimum of 2 NIC's. 1 NIC is used for the Service Console and 1 NIC for your VM Network. If you want redundancy, you need to install another Network Mezz card. The pipe between your Ethernet modules and your network will eventually limit your throughput unless you have multiple links from your external network.


Steven

[UselessUser]

Hi,

Thanks for that, I guess my next question is really what is the difference between all the modules available for the interconnect bays?

There are VC modules (Not really sure what these are for)
Vendor modules (HP etc)
Pass through

I assume pass through is just literally give a port to a blade (So no trunking or anything like that?)

[Rob_69_1]

Not completely sure (maybe somebody else can elaborate more on this), but the Virtual Connect modules should be more focused towards port trunking and should be more flexible when it comes to create NPIV logical network ports etc.

I would avoid passtrough modules, if you don't have very very specific needs.

Basically, as you already sayd, you'll have all the ports belonging to your blades EXPOSED outside as RJ ports (or FC ports). After that, you'll have to buy an EXTERNAL switch anyway, so you're going to spend almost the double. More then that, you'll have to cable FROM the exposed ports TO the switch, and then cable FROM the switch TO where you have to connect it. As you see, cabling will be a mess. All those cables will not be needed (communications will be inside the enclosure) if you buy switches that directly plug into the relevant connection bays (for instance in my C3000 enclosure I have a GBE2C switch module for ethernet connectivity and a Brocade 4GB FC switch module for FC connectivity towards the SAN).

HTH

[Steven Clementi]

"...I guess my next question is really what is the difference between all the modules available for the interconnect bays?"

- Selecting the appropriate interconnect is obviously an important step.

Virtual Connect Modules - There are various models for Virtual Connect. VC Modules help reduce the amount of cables you need to run to your enclosure while still providing some of the functionality you get with integrated switching. (High Performance Uplinks, Port Aggregation, VLAN Tagging, etc.)


Vendor Modules - Smaller versions of industry standard networking devices. These devices usually run the same exact firmware/software as their bigger brothers.


Pass through Modules - Simplely extends the connections from the servers to the outside of the enclosure. If you have 8 servers, each with 2 embedded nics... the passthrough has 16 RJ-45's on it. Each port corrosponds to 1 port on a server.


Selecting the right module for your environment depends upon the configuration you want.

Some of my customers use passthrough modules to directly connect to their core infrastructure in order to provide for faster (less hops) on their vMotion network(s) between enclosures and/or sites.


Steven