topic Re: RHEL 4.8 bnx2x drivers with bonding in Operating System - Linux

RHEL 4.8 bnx2x drivers with bonding

Rob Leadbeater — Fri, 16 Jul 2010 12:24:30 GMT

Hi,

BL460c G6 with dual onboard 10Gigabit Broadcom NetXtreme II NICs (NC532i) and dual Intel Gigabit mezzanine card (NC360m), running RHEL 4.8.

I'm trying to bond one of the onboard NICs (eth0) with one of the mezzanine NICs (eth2), however whenever I do so, I fail to get a link on the onboard card.

ethtool eth0 reports "Link detected: no"

If I remove the bonding configuration and just use eth0 the NIC works fine.

The attached output from messages shows what happens when the bond tries to start up. I'm guessing this is indicating a potential firmware/driver issue.

I've trawled through all the Advisories for this kit, but haven't found anything relevant, yet.

Firmware has been upgraded to the latest, (FW DVD 9.0. PSP 8.50 is installed. Switches in the blade chassis are Cisco 3020s in case that's relevant.

Any ideas ?

Cheers,

Rob

Re: RHEL 4.8 bnx2x drivers with bonding

Steven E. Protter — Fri, 16 Jul 2010 13:58:03 GMT

Shalom Rob,

Sadly I don't think the OS supports bonding this Interface yet.

I have hunted for documentation all over the place and come up empty.

SEP

Re: RHEL 4.8 bnx2x drivers with bonding

Rob Leadbeater — Fri, 16 Jul 2010 15:41:07 GMT

Hi SEP,

I hope that isn't the case...

I logged an SR with Red Hat earlier. Hopefully that will turn something up...

Cheers,

Rob

Re: RHEL 4.8 bnx2x drivers with bonding

Rob Leadbeater — Mon, 19 Jul 2010 08:09:27 GMT

Hi all,

Anybody else out there managed to run this configuration ?

I've not got a reply from Red Hat yet so I've also just logged a call with HP...

Cheers,

Rob

Re: RHEL 4.8 bnx2x drivers with bonding

Matt Palmer_2 — Tue, 20 Jul 2010 11:31:01 GMT

Hi,

I had the same issue with SLES10. I had to get different initrd to load in from RDP from HP site and also download the NetXtreme source drivers and compile from source(you will also need kernel-source and kernel-syms). Then add alias ethx bnx2 inside modprobe.conf

I think you will have to change to bnx2x if its a G6 460.

hope that points u in the right direction

regards

Matt

Re: RHEL 4.8 bnx2x drivers with bonding

Rob Leadbeater — Tue, 20 Jul 2010 11:47:54 GMT

Hi Matt,

Grabbing and building the latest drivers from Broadcom was next on my list of things to try... Looking at the release notes, they appear to be pushing out fixes every other day - only a slight exaggeration !

One thing I neglected to mention earlier is that I'm trying to use VLAN tagging on top of the bonded interface, as is quite common in blade servers. I did however find this gem in the Broadcom User guides:

"VLAN support is provided according to IEEE 802.1Q and is supported in a teaming environment as well as on a single adapter. Note that VLANs are supported only with homogeneous teaming and not in a multivendor teaming environment."

This may well be the issue as I have Broadcom and Intel NICs in the team...

Cheers,

Rob

Re: RHEL 4.8 bnx2x drivers with bonding

Matt Palmer_2 — Tue, 20 Jul 2010 11:52:19 GMT

Hi Rob,

I've just been through this pain barrier following a chassis migration because of core network replacement, PCI-DSS and VLAN tagging!! ;-)

are you vlans not working at all?

how have you configured your switches, and are they GbE2c's??

I can send you a dump of my switch config if it would help, although you would have to tailor some of it for your own use.

regards

Matt

Re: RHEL 4.8 bnx2x drivers with bonding

Rob Leadbeater — Tue, 20 Jul 2010 12:01:39 GMT

Hi,

We're using Cisco 3020s.

The VLAN tagging works fine on any non-bonded interfaces, so the switch configuration looks to be OK - I don't have access to the switches myself unfortunately.

If I get rid of the VLAN tagging from the bonded interface, then the link on eth0 does appear to come up. I'm waiting on the network guy to change the switch port configurations to be access ports, rather than trunk ports to see if that will sort things out...

Cheers,

Rob

Re: RHEL 4.8 bnx2x drivers with bonding

Rob Leadbeater — Tue, 20 Jul 2010 12:39:13 GMT

Success.

Getting rid of the VLAN tagging got things working.

Cheers,

Rob