NIC Bonding on RHEL 4

[Ross Kennedy]

Hi
I'm trying to set up Bonding on a DL380G4.
OS is RHEL 4 with the latest patches plus the latest HP Softpaq's (7.30).

While the box is still on the network, I don't think I have bonding set up properly (if at all) as only eth0 is "live". It doesn't fail over to eth0 if I disconnect eth0.

Before attempting bonding, I did check that both NIC's do work!

I've attached a file with my config files and the output from ifconfig. Somewhere I read that ifconfig should show bond0 as well as eth0 and eth1 but on my system ifconfig only shows bond0.

I've lost count of the number of manuals I've read. Also trawled through this forum and everything looks as though bonding is set correctly.

Can anyone suggestion how I go about troubleshooting?

[Patrick Terlisten]

Hi Ross,

look at this Bugzilla Report:

https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=159500

Other question: Can you reach the server with ping or ssh? Is the server available on the network? Can you bring eth0 and eth1 up with "ifup eth0" or "ifup eth1"?

Regards,
Patrick

[Stuart Browne]

Hrm.. Start by trying the HP supplied 'bcm5700' driver instead of the RH provided 'tg3' driver ( http://h18000.www1.hp.com/support/files/server/us/download/22318.html ).

If that still fails, also look at the 'HP tested bonding driver' ( http://h18000.www1.hp.com/support/files/server/us/download/22271.html ).

Also, what sort of switch do you have the other end of the network cables plugged in to? Have you set up (or does the switch auto-detect) trunking for the two ports?

[Ross Kennedy]

Thanks for the reply.
I can connect to the box quite happily. PING and SSH both work.

Here is what happens when I run ifup.

[root@penguin ~]# ifup eth0
tg3 device eth0 does not seem to be present, delaying initialization.
[root@penguin ~]# ifup eth1
Enslaving eth1 to bond0

See attachment

eth1 makes an appearance in ifconfig (good) and netstat (which isn't what I would expect) so you are on the right track in that eth1 appears to start disabled. I can see in netstat -i that the RX count for eth1 is increasing but the TX count is not changing.

However, the bonding function still isn't working. To me, it looks as though I have 2 NIC's called bond0 and eth1 with the same IP and ethernet addresses but they are not bonded.

[Ross Kennedy]

Stuart,
Thanks for the reply.

The system is in our lab just now. Both NIC's are cabled into a hub which is taking care of the age old problem of HP/Compaq NIC's failing to auto detect 100M/Full on our switches. ethtool shows both bond0 and eth1 have autodetected to 100M/Full OK.

I've checked that I DO have bcm5700 installed.
[rkennedy@penguin ~]$ rpm -q bcm5700
bcm5700-7.4.12b-1

I confess I've tried changing modules.conf but it doesn't seem to make any difference but I'll definitely go back to alias eth0/1 bcm5700.

I did initially try to install the bonding driver but read somewhere that bonding is natively supported on this server&OS combination.

[Steven E. Protter]

This is very problematic on Linux, and it gets worse if you use Gigabit cards.

The only GB cards I've gotten to work in the RedHat/Centos world is Intel. You have to add code to your /etc/init.d/network script to make the bonding work.

Also if you use ethtool after doing the bonding, you will find the speed does not show accurately.

If you want specefic configuration files, use this webform to contact me.

http://www.isnamerica.com/contactsep.shtml

SEP

[Daniel Hili]

Hi Ross,

Bonding is supported natively by RHEL and I have tested with both tg3 and e1000 drivers.

I've recently had a lot of problems configuring two e1000 NICs as bond0 on an RHEL 3 system.

We could successfully load up the bonding module using modprobe whilst the operating was up and running but for some unknown reason, after every reboot the NICs would start up wrong and we wouldn't see all devices listed properly. Using ifconfig -a we would see NICs listed as devXXXXX rather than the usual eth0, eth1, etc.

Anyway to keep a long story short, we renamed our script /etc/sysconfig/network-scripts/ifcfg-bond0 to ifcfg-zbond0. Adding the "z" effectively changed the starting lineup of the bonding interface to after the slave interfaces. Ever since we made this change bonding has worked a dream but I've still got a call logged with RH to figure out what is going on.

Hope it helps.

Dan

[Matt Palmer_2]

Hi,

have u looked here:

http://docs.hp.com/en/B9903-90046/ch05s04.html

that is redhat specific, I have got this workin g successfully on SuSE SLES8, not sure if there is much difference in the procedure, but one problem I had was that the instructions were slightly back to front in places, namely adding the bond lines to the /etc/modules.conf doc before trying to use ifenslave to enlist the specific NICS to the bond0.

HTH

regards

Matt Palmer

[Matt Palmer_2]

Hi, the other thing I forgot to mention is that installing the proliant support PAQ (PSP) is quite handy, as you can see in a user friendly format, how your cards are working, ie: if you've set the mode= line to '1' then the SIM web front end(http://serverip:2301) should tell you under NIC that bond0 is up using active-backup mode, if you set mode to 3 then it will say, 'switch assisted load-balancing, and if 2 'balanced-xor'.

One thing to note is that mode 3 does not work well if both NICS are going into the same network switch

one last thing is did u rebuild the kernel when you did this to get the bonding rpm or did you use another method?

regards

Matt Palmer

[Stuart Browne]

To counter SEP's issues with gigabit cards, I've had success with the Broadcomm series that are in the HP server range.

Just don't have any here to play with at the moment.