Weird Ethernet problem - randomly losing link during boot

[W Sanders_1]

This is more on a FYI in the hope that it will save someone else from having to install a new system board. We have some DL380 G4's running RHEL3, I'm pretty sure it has not been patched recently, and since I made some changes to the eth0-config file that seemed harmless, one or the other or both eth interface would lose link and not come back during bootup.

We are in an environment where our sadly misinformed "network support" team does not allow us to use autonegotiation, so we have to try to fix our ports at 100-FD. (If autonegatiation is left enabled on the HP and the Cisco port is fixed at 100-FD, the HP will come up as 100-HD which will work, but collisions occur and get counted as errors, and this also bothers the "network support" team.)

In an attempt to work around this, I put this line in eth0-config:

ETHTOOL_OPTS="autoneg off speed 100 duplex full"

This semeed to have no effect, but as soon as we rebooted the machine we started having problem with eth0 AND/OR eth1 randomly dropping link during the boot process. Sometimes it was eth0, sometimes eth1, sometimes both would not come up. I removed the ETHTOOL_OPTS from eth0-config and everything seems to work. even though eth1 was affected, there was never an
ETHTOOL_OPTS in eth1_config.

"Mii-tool" seems to successfuly force the interface to 100-FD when executed in S99local. Surprisingly, it doesn't seem to bring the link down to switch the duplex late in the boot process.

The tg3.o included in RHEL3 is an old one - from strings a see "tg3.c:v2.2 (August 24, 2003)" in the driver. I had a devil of time with this driver on Dell Poweredges until about this time last year when I upgraded to 2.4.24 (?) kernels under RedHat 9.

Conclusion - Maybe upgrade to bcm5700 if you MUST defeat autonegotiation, or upgrade to RHEL4.

[kcpant]

Hi Sanders,

Good story, but I would never like to change the default behaviour ( auto negotiate) of eth.

[lahteenj]

Hi We have also 380 series servers and cisco switches.

What should help is that you tell the network support team to look in cisco logs on those ports and look if the cisco tires to autonegaotiate different speeds. That has been the case here, but this has only been a problem on older server.

I think the easiest solution is to configure the Cisco to the same as speed and duplex as the server. That has always worked for me.

We never use autonegiotiate on either Cisco or Servers.