Virtual Connect and server NIC Teaming question

[DerekS_1]

We have BL490c G6 servers with two C-7000 chassis that use Virtual Connect Flex-10 modules. I've configured the VC module shared uplink sets for active/active mode. Two external switches are Nexus 5010. The chassis are in a single domain and are stacked.

For each SUS that I need I created two SUSes, one using the external ports on chassis 1 and the other SUS for the ports on chassis 2.

On a particular server I use mapped VLANs. NIC 1 would get VLAN 10 from SUS1 and NIC 5 would get VLAN10 from SUS2. Now I'm not sure what teaming mode I should use for Windows Server 2008 R2 and ESXi 4.0 hosts.

Given the Nexus switches can team across switches, I'm thinking that might give me more options? I need to know the proper teaming mode for 2008R2 and ESXi that hopefully can send/receive traffic down both Flex-10 NICs.

[The Brit]

It isnt clear from your post how many VC modules you have in your enclosures. I even sounds like you might have only one in each enclosure (not good).

If you do have pairs of modules, the recommended way to set up Shared Uplinksets is that you shouldnt combine ports from different modules into the same uplink set.

The main consideration when setting up NIC teams is to keep in mind that c7000 enclosure hardware redundancy is set up between horizontal pairs of Interconnect Modules, (which is why the ports on individual cards (MEZZ or LOM) are mapped to horizontal pairs of interconnect modules). Think of it as a Left versus Right setup.

The easiest situation to think about this is the situation where you are doing a firmware upgrade. VC is smart enough to ensure that at least one module in a horizontal pair is always online. This pretty much guarantees that if you NICs are teamed between horizontal pairs (i.e. one port on the left, and one port on the right) then you will always have connectivity.

If you team between vertical pairs (i.e. both ports on the left, or the right, then VC does NOT guarantee that either of the modules will ba available during a Firmware upgrade.

Note: para 2 is a recommendation, not a rule. I see evidence that there are alternative ways of setting up SUS's depending on local requirements or preferences.

Para's 3, 4 & 5 however are physical realities and cannot be avoided if you want proper redundancy and network failover.

HTH

Dave

[DerekS_1]

Let me clarify a few things. Each C-7000 has two Flex-10 modules. Each SUS only has ports from one module associated with it. Each SUS is assigned two uplink ports, which VC shows as active/active since I have LACP enabled on the Nexus switches.

The two chassis are stacked. Each Flex NIC is assigned one or more mapped VLANs. Flex NIC A is only assigned VLANs from SUS1, and NIC B is only assigned VLANs from SUS2.

SUS1 uses uplinks from the left module on chassis 1. SUS2 uses the right module from chassis 2. Each chassis is connected to a different external Nexus switch.

My question is regarding what NIC teaming mode I should use for Windows Server 2008 R2 and ESXi 4.0. Can I use switch assisted load balancing for Windows? Can I use "ip hash" based balancing in ESX?

Since I can't configure load balancing on the Nexus switches beyond the LACP for the SUS links, I don't think it makes sense to use a OS load balancing option that requires switch assistance (port channel load balancing).

[The Brit]

Hi Derek,

According to your last post,

"SUS1 uses uplinks from the left module on chassis 1. SUS2 uses the right module from chassis 2. Each chassis is connected to a different external Nexus switch."

The left module on chassis 1, and the right module from chassis 2, do not make up a horizontal pair, and there fore there is no guarantee that one of them will always be up.

I would suggest that with this configuration you are still at risk of experiencing conditions where there are NO SUS's available, and therefore NIC teaming become irrelevant.

Could be wrong though!

Need a tie breaker here...

Dave.