Problem with Flex-10 NIC team accessing internet

[JonathanT]

I'm having trouble with a NIC team accessing the internet. So here is what I have... a BL490C G6 blade with a FlexNIC team. This Team is connected to a VC Flex-10 pair of interconnects. Those are cross connected to a pair of VC 1/10 virtual connects. Those uplink to a pair of Dell PowerConnect 6224 switches (stacked). The Powerconnect 6224 connects to a Dell Powerconnect 2448 (simpler layer 2 switch). The Powerconnect 2448 connects to a Cisco RVS4000 that accesses the internet. For simplication purposes I have drawn this diagram:

FlexNIC 1 + FlexNIC 2
| |
VC Flex10 A - VC Flex10 B
| |
VC 1/10 A - VC 1/10 - B
|
PC 6248 A - PC 6249 - B (stacked)
|
PC 2448
|
Cisco RVS 4000
|
Comcast Cable Modem

The problem I'm having is when the NIC team is in place I can't hardly do anything with the internet. I can access everything on the internal network no matter which switch it's connected to (other blades, servers off 6248 and servers off 2448). I've verified that jumbo frames are not turned on for the NIC team. The team was originally configured as a TLB load balancing. If I switch that to fault tolerance load balancing than everything works fine. Is this some kind of issue with needing to turn on spanning tree (RSTP) on for the RVS4000 or something if I'm in a TLB scenario?

[HEM_2]

HP NIC Teaming - TLB does IP address spoofing. Maybe your Cisco RVS4000 device does not support that.

When teaming 2 NICs together in TLB, one is designated as the primary which will TX and RX and one is designated as the secondary which will TX Only. All communication to the NIC team will go to the TX/RX NIC's MAC Address. When the NIC team is sending traffic out it will balance the traffic between the TX/RX NIC and the TX Only NIC using various algorithms. When the TX Only NIC is used for transmit, it will use its own burnt-in MAC Address as the source address for those frames instead of the NIC Team MAC Address. Maybe the RVS4000 device sees this communication as a security violation OR maybe the RVS 4000 is trying to respond back to the TX Only MAC Address (which the NIC Team will ignore).

THere is probably not much you can do to resolve this other than to not use TLB or maybe there is some settings you can manipulate on the Cisco RVS 4000 device.

[JonathanT]

Is there a better load balancing algorithm that I should be selecting for the NIC team that will allow it to receive frames on the 2nd NIC just in case the traffic is returned there by the RVS4000? I know that with the Flex-10 being two separate switches that there is no way to load balance and get 2X RX like you could etherchanneling to a single Cisco switch.

What are you guys using for your NIC teaming algorithm on FlexNICs?

[JonathanT]

I worked on this issue some more this morning. Upgrading the NCU (and the required NC532i miniport driver) had no effect on the problem. I was however able to figure out one thing. The server can communicate just fine to the internet if itâ s traffic passes through a Cisco ASA 5510 firewall. The firewall is NATing the traffic from 192.168.1.X network to 192.168.100.X network. The Cisco ASA 5510 passes the traffic off to the Cisco RVS4000. If I put the blade directly on the 192.168.100.X network and pass the traffic to the Cisco RVS4000 appliance it causes a problem. So it appears to be an issue between the TLB and the Cisco RVS4000. Any ideas on how to troubleshoot this or something I might need to change on the Cisco RVS4000?

[HEM_2]

I'm not sure of what settings might help on the RVS4000.

You might try a static IP to MAC address entry.

Map the IP address of the team to the team MAC Address on the RVS4000. That might work.

[JonathanT]

Worked on this all day and got no where. Of course the Cisco SMB support had no clue of high end hardware like the Flex-10. The HP support had no clue about the RVS4000. Nothing fixed the problem. I guess I'll have to settle for NFT instead of TLB in this scenario. If HP wants to spend $150 on an RVS4000 they could troubleshoot this for me :)

[GZamboni]

I don't know yet, but, changing the TLB method to Destination MAC Address worked for our servers.

[Adrian Clint]

One thing to be aware of.
If you are connecting to stacked switches and you have a port trunk/lacp/etherchannel on the stack but going to more than one VC module then it will only make active the paths to one VC module. The others will be standby.
You cannot port truck accross two different VC.
You have to create two different port trunks - one to each VC.

[JonathanT]

That is exactly correct. So I have 1 4Gb trunk to the core active at a time. If one core switch fails then the other 4Gb standby trunk takes over.

[shocko]

TLB teams do not do IP spoofing. i dont know wher you get that information. make sure there is no IP FastPath option on your upstream switches as this can cause serious problems with TLB teams. vrutual connect doesn't need spanning tree.