c7000 NIC port mapping with Ethernet VC Modules.

[The Brit]

Hi guys,
This is probably one of the commonest questions regarding VC, however I am just trying to crystalize my understanding of the "theory", before my first systems arrive (in the next couple of weeks).

I have a couple of fairly simple questions, (I think), the hardest part is trying to phrase them correctly.

1. Lets say I have 1 c7000 enclosure containing 8 (full height) BL860c blades.
2. I have 2 Ethernet VC modules in Interconnect bays 1 & 2.
3. Each blade has 4 integrated NIC ports, (NIC1, NIC2, NIC3, & NIC4)
4. Each ENet VC module has 8x1Gb uplink ports (lets say P1-P4 on top, and P5-P8 at the bottom (we will ignor the 10G (CX4) ports for the moment).

My understanding so far is,

a. The Ethernet VC modules in IC bays 1 & 2 represent a redundant pair, and cannot be both active at the same time.
b. NIC1 and NIC3 on each blade are hard mapped to IC bay 1.
c. NIC2 and NIC4 on each blade are hard mapped to IC bay 2.

Question 1.

Is my understanding correct so far?? In particular, a. referring to whether Bay’s 1 & 2, are Active/Passive, or can they be configured as Active/Active, and if so does this have implications regarding redundancy. Put another way, with this configuration, can I have 16 active uplinks (with 2x8-port Ethernet VC modules), or can I only have 8 ??

Now if we consider just NIC1 (on all blades). They are all mapped to the module in IC bay 1.

Question 2.

Are they mapped to any particular uplink port on the VC module, or any set of ports ?? If the answer to this is NO,

Question 3.

How do I control/segregate/split the network traffic between redundant external LANs/VLANs ?? (I assume this is configured through the OA)

Thanks in advance for any help you can provide.

Dave

[Phil uk · Accepted solution]

Hi,

I'm not "crystal" on all of this myself, but my understanding is that if you had for example 2 VC modules, one in Bay1 and the other in Bay2, then both would be "useable" - ie, have ports that you can use for uplinks etc, but the VC in Bay1 would be running the Active VC Manager from which you do all the configuration work. So, should that VC Module in Bay1 fail, then the VC Module in Bay2 would become the Active VC Manager.

With regard to control/aggregating/splitting LANs and VLAN's etc, this is all done via the VC Manager GUI which you can access through the OA. But you only want to start configuring the VC Domain once you have setup all the IP Addresses for the OA's, ILO's and various modules that you may have.

Then you can start the VC Manager GUI and create the VC Domain and configure your server profiles / LAN configs etc.

Hope this helps
Phil

[The Brit]

Thanks Phil,
Your response is a good start. It highlighted the idea that the redundancy (failover?) is between the VC Managers and not between the VC Modules themselves.

I guess that means that the 16 uplink ports in the two modules are all available and can all be active.

Is it then up to me to configure the ports to cover the possibility that there could be a VC module failure. For example, lets say that my blades represent an OpenVMS cluster, and on each blade I allocate

NIC1 --> Used for general TCPIP network traffic via VLAN 1.
NIC2 --> Used for LAT traffic via VLAN 2.

Now my understanding is that NIC1 is mapped (in the midplane) to IC Port 1 (and therefore to VC Module 1). Now if VC Module 1 fails, the VC Manager fails over to Module 2 (in IC port 2) but what happens to the communication which is moving through NIC1. It is hard wired to IC port 1, which now contains a FAILed VC Module.

Am I screwed here ?

Dave

[sterls27]

Hold on here...the Ethernet modules are NOT in a state that they are redundant and only one can be used at a time - you can configure them that way but it's a complete waste of resources. All ethernet modules are configurable/accessible at the same time. The point of the Virtual Connect domain is redundancy. If any one of those modules fails, VC will find the best way to get back out to the network. There are no implications to having all modules as active/active. The CX4 cables (if connected properly) add redundancy to the redundancy. The blade profiles that are configured within the Virtual Connect domain also provide redundancy should the hardware fail, now granted it only maintains your original network/fiber settings. It does not keep a copy of your server. :) All the blades are mapped to particular ports, but they will use what you assign them to use within the VC Domain (the blade profiles).

As far as Question 3, this gets a little tricky. One thing that MUST be constant is the VLAN that the chassis and iLO's are configured on. You cannot have the ethernet modules on say 192.168.1.x and the OA on 192.168.2.x, they need to be able to communicate with each other and thus all must be on the same VLAN. You can designate whatever VLAN you want for the BLADES to use within the Virtual Connect domain, not the OA. All network configurations are done within the Virtual Connect domain. You can have the entire chassis on 192.168.1.x and the blade profiles configured to use 192.168.2.x as their network connections within the OS on the blades. The best configuration you want for redundancy on the ethernet modules, is to have enet1 connect to switch 1 and enet2 connect to switch 2. That way if switch 1 goes down, Virtual Connect will know to redirect traffic through enet2 since it is able to reach switch 2.

I have some documenation from when we set up our first C-Class chassis that I can forward to you and should help answer most of your questions. This new set up can be a little confusing, but I have found it to be wonderful once you have it set up and working.

[The Brit]

Thanks for the help guys.