topic Re: NIC Teaming with two GbE2c switches in C3000 in BladeSystem - General

NIC Teaming with two GbE2c switches in C3000

John van_2 — Tue, 20 Jan 2009 21:56:17 GMT

Hi,

I have a BL460c server with an additional Mezzanine NIC card. My C3000 enclosure has two GbE2c switches.

I have set up a HP Network Team with one port from the embedded NIC and one port from the mezzanine NIC to provide a redundant link through both switches to the LAN.

The team connects but constantly drops one port (embedded) with status "Failed (Rx Path)". It repeatedly tries to pick up the second port but then reverts to the status identified above.

Any suggestions as to what I might have done wrong?

Thanks,

John

Re: NIC Teaming with two GbE2c switches in C3000

Ken Henault — Tue, 20 Jan 2009 22:35:21 GMT

What OS is this for? What teaming method did you select?

Re: NIC Teaming with two GbE2c switches in C3000

John van_2 — Tue, 20 Jan 2009 22:48:28 GMT

Hi Ken.

Windows 2008 X64

Used the HP NCU
Team Type Selection : Automatic
Transmit Load Balancing Method: Automatic

Is that what you were after?

John

Re: NIC Teaming with two GbE2c switches in C3000

Lmm_1 — Wed, 21 Jan 2009 14:47:14 GMT

Do you have crossconects enabled (Port 17 and 18)? can you run a Network survey? from command propmt: hpnetsvy >filename.txt

rgds,

Re: NIC Teaming with two GbE2c switches in C3000

John van_2 — Thu, 22 Jan 2009 01:39:11 GMT

Yes. 17 and 18 are enabled. The survey details are attached. I have removed the details of Team 1 which is an iSCSI team connected to a SB600c storage blade.

John

Re: NIC Teaming with two GbE2c switches in C3000

Miika T — Sat, 24 Jan 2009 13:08:09 GMT

I'd change the nic teaming to NFT + Transmit load balancing. As the team is split between to swithes you can't select switch assisted LB, which can happen with automatic setting.

-MiikaT

Re: NIC Teaming with two GbE2c switches in C3000

John van_2 — Sat, 24 Jan 2009 22:21:33 GMT

Hi MiikaT.

I tried your suggestion to set NFT. The load balancing greys out but I assume it still does load balancing. Unfortunately, the problem remains. After a second or two one of the NIC's is disabled. The event log actually says "All but one is disabled". Because there are only two, it means just one of the two NIC's is disabled.

I have found a reference which says Tree Spanning should be disabled on the switch ports to which this server connects. I haven't been able to work out how to disable Tree Spanning on individual switch ports on the GbE2c. I can only find where I can disable Tree Spanning by VLAN.

Would you recommend I try that?

Thanks,

John

Re: NIC Teaming with two GbE2c switches in C3000

Ken Henault — Sun, 25 Jan 2009 13:37:25 GMT

That is the correct behavior. NFT is active/passive teaming. Both NICs need to be connected to the same switch for active/active teaming.

See this white paper for a better understanding of the available options:

http://bizsupport.austin.hp.com/bc/docs/support/SupportManual/c01415139/c01415139.pdf

Re: NIC Teaming with two GbE2c switches in C3000

Lmm_1 — Mon, 26 Jan 2009 03:54:44 GMT

The recommended port configuration is port-fast enabled (Cisco terminology), here is the gbe2 equivalent:

>> # /cfg/l2/stp 1/port 20
>> Spanning Tree Port 20# fastfwd ena
>> Spanning Tree Port 20# apply
>> Spanning Tree Port 20# save

I would try to force the team mode to TLB. By default Port 17 and 18 are members of Trunk group 1, make sure trunk 1 is enabled and can carry traffic of the VLAN where the Ports connected to the teamed NICs belongs.

Re: NIC Teaming with two GbE2c switches in C3000

John van_2 — Tue, 27 Jan 2009 03:15:05 GMT

I applied fastfwd to the ports connecting the blade server to the switches. These ports are in VLAN1. Ports 17 and 18 are members of Trk1 which is enabled. 17 & 18 are currently not in VLAN1. I also have VLAN10 which is the iSCSI VLAN connected to the SB600c through the other port from each NIC. Ports 17 & 18 are in VLAN10. If I put those ports in VLAN1 as well then the LAN Team works OK but the iSCSI Team fails.

I thought ports could pass traffic for more than one VLAN. I'm guessing I need to change configuration to allow that to happen, but I'm not sure what that may be.

Thanks,

John

Re: NIC Teaming with two GbE2c switches in C3000

Lmm_1 — Tue, 27 Jan 2009 13:32:44 GMT

You need to tagg those ports and put them on both VLANÂ´s, basically make it work as 802.1q port... it might be something like this for each port on trunk 1:

>> Main# /cfg/port XX
>> Port XX# tag e

>> /cfg/l2/vlan XX
>> VLAN XX# add XX(Port)

then apply and save, with this config port 17 and 18 will be able to talk on VLAN 1 and 10.

Re: NIC Teaming with two GbE2c switches in C3000

John van_2 — Wed, 28 Jan 2009 08:30:17 GMT

Thank you very much. Applying tagging to VLAN's on ports 17 & 18 fixed the problem. I could then add the ports to both VLAN's.

I have set both the iSCSI and LAN Teams to TLB with fault tolerance. I have left the TLB method as Automatic. Is that what you would recommend?

Thank you very much for your replies and also thank you to the other respondents.

John

Re: NIC Teaming with two GbE2c switches in C3000

Lmm_1 — Wed, 28 Jan 2009 13:24:15 GMT

In this case (Teaming embedded with mezz on a c3k), the best option will be TLB.

Re: NIC Teaming with two GbE2c switches in C3000

John van_2 — Wed, 28 Jan 2009 21:36:20 GMT

Thank you again. A couple of final questions if I can. Should I Port-Fast enable the teamed ports connected to the iSCSI VLAN as well as the LAN VLAN?

I also have 2 blades running VMware ESX 3.5 with a similar setup i.e. teaming one embedded and on mezzanine port together. This may be outside your area of expertise, but should I apply Port-Fast enabled to those ports as well?

John

Re: NIC Teaming with two GbE2c switches in C3000

Lmm_1 — Wed, 28 Jan 2009 21:42:27 GMT

You should aply portfast to any port connected to a host in order to avoid STP delays (Listening, learning...etc)... OS doesnt make any difference, if you got different vlans on a vswitch, the port where the VMNIC goes ,must be tagged.

Re: NIC Teaming with two GbE2c switches in C3000

John van_2 — Wed, 28 Jan 2009 23:21:30 GMT

Thank you. I am now comfortable with the way my system operates after following your suggestions.

John