NIC Teaming/Switch Configuration

[Kris Nawrocki]

I am having network issues with my HP servers (dl360 g3's). I have the HP teaming software (latest ver.) installed and have the two nics teamed using "Load Balancing w/Fault Tolerance" and use Transmit Load Balancing and Balance with MAC address for controls.

Each NIC is connected to a different switch for redundancy.

The problem is that sometimes a server is unreachable through pings and other connectivity tests. It will suddenly drop off the network. Then eventually it will be reachable again.

Anyone ever ran across this problem? Perhaps i am using the wrong teaming method or the switches have to be configured a certain way.

thanks for any help.

[SAKET_5]

Hi Kris,

I believe something is misconfigured...

First of all, why do you balance your team based on MAC address, you do realise that this will point to the exact same MAC address for all traffic originating from behind a router as it will be the MAC address of the router. Rule of thumb here...When an HP Network Adapter Team needs to load balance traffic that traverses a Layer 3 device (Router), IP Address load balancing should be used.

Could you first confirm this for me..?

hope this helps and dont forget to assign points:)

regards,

[SAKET_5]

Hi Kris,

Just realised that in your original post you mention for switch redundancy, you have connected the NICs to two different switches?

Please note that HP Network Adapter Teaming article makes it very clear that all switch ports that are attached to members of the same team mmust comprise a single broadcast domain (i.e., same VLAN).

Additionally, in your scenario where you are experiencing intermittent network connectivity problems with Network team, the first thing I would do is to move the NICs to the same switch. If this solves your problem, then the issue lies in the configuration of the switches and nothing should be wrong at the server end.

hope, this helps and dont forget to assign points:)

regards,

[SAKET_5]

Kris,

keep coming back...

Are spanning tree's blocking, listening and learning stages disabled or bypassed on the all the switch ports to which your Network Adapter team is attached. If not, you must do this.

Also, according to the HP NIC Teaming guide, if switch redundancy is required, then HP recommends that the switches be deployed with redundant links between them and spanning tree be enabled on (or other layer 2 redundancy mechanisms) on the ports that connect the switches.

I believe following the above guidelines should bring an end to the problem. We successfully manage hp proliants with a number of them with teamed NICs - No probs at all.

Final point which I am sure you are aware of ...keep your system BIOS, NIC's FW, Proliant Support Pack to the latest version.

hope it helps and let us know how you went..dont forget to assign points:)

regards,

[Thomas Bianco]

i think the option "Load Balancing w/ fault tolerance" uses LCAP*, which must be supported by the switch (singular) that the server is attached to.

i think you want "switch fault tolerance" which uses an active/passive teaming to account for a failure of a single card or the switch it's attached to. there is no load balancing with switch fault tolerance, because there is no way to aggrigate across switches.


*Link Control and Aggrigation Protocol.

[Thomas Bianco]

poke to the devs: it would be nice if the option that used LCAP was called LCAP.

ps: as this is not related to your problem, please 0-point this post.

[SAKET_5]

Kris,

any updates?

[Derek_31]

In our environment of Cisco 6509 switches, we use Fault Tolerant connections when teaming and connecting one server to two ethernet switches. We only use load balancing when the both ports are on the same switch.

My network guys tell me that Cisco can't bond two ports from two different switch to aggregrate the bandwidth. IOS only lets you do that within a switch chassis.

[Kris Nawrocki]

Thanks for all the suggestions. I am still waiting to try some of these because these are production machines and i have to schedule the downtime. thanks

points to come.

[Khoa Pham_2]

Gang,

I am the network engineer working with Kris to resolve this issue. Allow me to give more information about the network side.

We have two Foundry fastiron workgroup switches (Layer 2) that are cross connected and running spanning tree. There are two connections, one is blocking and one is forwarding. Both switches are in the same broadcast domain.

We are seeing a behavior where the switch that is connected to the primary NIC cannot ping the host however the secondary switch can. As Kris said, this is an intermittent problem. We also had Foundry SE's investigate but they didn't know enough about HP teaming to really give us a definitive answer.

Any other suggestion are appreciated.

[Derek_31]

This is a classic symptom of using load balancing when you should be using fault tolerant configurations.

[Thomas Bianco]

this is the same behevior we saw durring an the migration from LCAP connections (i.e. Load Balanced) with a cisco and Intel Teaming Software to multiswitch fault tolerance.

one the teaming was swiched to Switch fault tolerance (i.e. hot standby) the problem disappeared.

[Derek_31]

Cisco does not support, per my network engineers, bonding of ethernet channels across multiple switches. This is why you must use fault tolerant mode when teaming across multiple switches. I don't know if other ethernet switch vendors have the same limitation.

[SAKET_5]

Kris and Khoa,

Spanning tree needs to be disabled on the switch ports to which your teamed NICs are connected to. Also, please ensure that the teamed NICs are connected to the same switch or if in the case of multiple switches (same broadcast domain tho), ensure spanning tree is enabled on the ports which connect the two switches.

I totally disagree with previous comments about how you should only use teaming for fault tolerance. Why not load balance, sure you may or may not get a true link agrregation or fat pipe, but we do use it for load balancing....absolutely nothing wrong with that!

hope it helps and could you assign points to the replies please...

regards,