Link down messages once Broadcom drivers are loaded during bootup

[Joseph Cipolla]

I recently patched a DL560 machine from RHEL 3 update 4 to RHEL 3 update 5. After the process was complete, I rebooted the system to use the 2.4.21-32.0.1.ELsmp kernel. During the boot process eth0 and eth1 (which are the on board broadcom interfaces) show up as OK. However, once I get beyond that point I see the bcm5700: Link Down message on both interfaces.

I was able to get them to work by downing both interfaces, running a modprob -r bcm5700 to remove the driver, and then reinstalling the module by running insmod bcm5700. It detects both interfaces and they come right up. The modules.conf has the alias set for eth0 and eth1 to be bcm5700. Does anyone have any suggestions or further steps to take to resolve this issue? Thanks.

[Gopi Sekar]

I believe bcm5700 drivers are old and deprecated, use the new 'tg3' drivers.

update your modprobe.conf file with tg3 drivers and see whether it works for you.

Regards,
Gopi

[Joseph Cipolla]

The system was actually using the tg3 drivers first and I loaded the bcm5700 driver as a fix, but both were experiencing the same issue. The date according to Broadcom for the driver is 8-23-05. Is there a newer tg3 driver than this that I can acquire? Thanks.

[Gopi Sekar]

Check the Proliant support pack 7.40 version it should contain the updated tg3 driver if exists.

Regards,
Gopi

[Joseph Cipolla]

I only see the support pack version 7.30, I downloaded the bcm5700 driver individually and installed it. Here is the output from dmesg:

ip_tables: (C) 2000-2002 Netfilter core team
Broadcom Gigabit Ethernet Driver bcm5700 with Broadcom NIC Extension (NICE) ver. 8.2.18a (08/01/05)
divert: allocating divert_blk for eth0
eth0: HP NC7781 Gigabit Server Adapter found at mem f7ef0000, IRQ 11, node addr 000bcd0de167
eth0: Broadcom BCM5703 Integrated Copper transceiver found
eth0: Scatter-gather ON, 64-bit DMA ON, Tx Checksum ON, Rx Checksum ON, 802.1Q VLAN ON, NAPI ON
divert: allocating divert_blk for eth1
eth1: HP NC7781 Gigabit Server Adapter found at mem f7ee0000, IRQ 15, node addr 000bcd0de168
eth1: Broadcom BCM5703 Integrated Copper transceiver found
eth1: Scatter-gather ON, 64-bit DMA ON, Tx Checksum ON, Rx Checksum ON, 802.1Q VLAN ON, NAPI ON
ip_tables: (C) 2000-2002 Netfilter core team
eth0: Using PCI INTX interrupt
ip_tables: (C) 2000-2002 Netfilter core team
eth1: Using PCI INTX interrupt
bcm5700: eth0 NIC Link is UP, 1000 Mbps full duplex
Intel(R) PRO/1000 Network Driver - version 5.6.10.1-k2-NAPI
Copyright (c) 1999-2004 Intel Corporation.
divert: allocating divert_blk for eth2
e1000: eth2: e1000_probe: Intel(R) PRO/1000 Network Connection
divert: allocating divert_blk for eth3
e1000: eth3: e1000_probe: Intel(R) PRO/1000 Network Connection
ip_tables: (C) 2000-2002 Netfilter core team
bcm5700: eth1 NIC Link is UP, 1000 Mbps full duplex
ip_tables: (C) 2000-2002 Netfilter core team
e1000: eth2: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
e1000: eth3: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
lp: driver loaded but no devices found
usb.c: registered new driver serial
usbserial.c: USB Serial support registered for Generic
usbserial.c: USB Serial Driver core v1.4
bcm5700: eth0 NIC Link is Down
bcm5700: eth1 NIC Link is Down
mtrr: type mismatch for f6000000,800000 old: uncachable new: write-combining
mtrr: type mismatch for f6000000,800000 old: uncachable new: write-combining
e1000: eth2: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
e1000: eth3: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex

Like before, during bootup the screen shows OK for starting the interfaces, but then shows link down. The only way I have been able to restore the connections is when I do a modprobe -r bcm5700 then insmod bcm5700. This restores the link as normal. Am I missing something else here?

[Gopi Sekar]

bcm5700: eth0 NIC Link is Down
bcm5700: eth1 NIC Link is Down
mtrr: type mismatch for f6000000,800000 old: uncachable new: write-combining
mtrr: type mismatch for f6000000,800000 old: uncachable new: write-combining
e1000: eth2: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex
e1000: eth3: e1000_watchdog: NIC Link is Up 1000 Mbps Full Duplex


I believe something is happening here. How many Ethernet interfaces you have in this box? According to dmesg output there are 4 ethernet NICs (two broadcom and two Intel).

Also it shows that when broadcom link goes down, Intel nics are coming up!! pretty strange. I assume both are happening almost at the same time. Are you having some sort of load balancing configuration in your network? if it is so then the problem might be there. Also try disabling the intel NICs completely by commenting eth2 and eth3 (containing e1000) lines in /etc/modprobe.conf file, reboot the system and let us know how it goes.

Regards,
Gopi

[Joseph Cipolla]

I did what you suggested, eth2 and eth3 did not load or plumb up the interfaces. The same problem still occurred with eth0 and eth1.