Network device and driver issues with RHEL AS4 install on HP Blade Servers

[CA1367359]

Hello all

We have DL580 / BL20pG3 blades in our environment (with HP NC7170 Dual Port PCI-X 1000T Gb and 4xHP NC7781 PCI-X Gigabit NICs). When we install RHEL AS4 U2, strangely eth0 becomes eth3 and sometimes eth2 etc.. its really a trial-and-error with different network aliases / device names when using kickstart installs.

Finally we made it work, but our apps are running slow, we aren't sure if the network is properly configured /etc/modprobe.conf has the following entries.
alias eth0 tg3
alias eth1 tg3

Is it optimal to use tg3 driver for these network adapters? will it reduce the performance? Is there any HP standard driver to be used to get maximum performance for these devices?

Please advise.

[Jun Yu]

Hi, i've handle the similar problem before, suggest you to replace the NICs drivers with HP's.(download from hp driver and support site or from your smartstart support CD).

[CA1367359]

Thank you Jun Yu.

I have already installed Proliant Support Pack on all servers, so should I use e1000 as alias name in /etc/modprobe.conf.

Or should I need to install the drivers which updates the modprobe.conf?

If no drivers need to be installed, what are the best alias names to be used in the conf file? bcm5700 or e1000 ???

Also, a little surprising thing is mii-tool reports network cards speed as 10 Mbps but ethtool reports 1000 Mbps

Should we need to turn ON auto negotiation on the Server farm switches? ( We have Cisco 3750s)

Lot of questions!!!.. but hopefully someone can provide direction.

Thank you.

[Al Licause]

>>I have already installed Proliant Support Pack on all servers, so should I use e1000 as alias name in /etc/modprobe.conf.

You stated that you are now running the TG3 driver. This would indicate that you are using a Broadcom device in which case you would use the BCM driver. The E1000 driver is for Intel devices, I believe. Each driver should have more complete information in it's accompanying README and/or other text files.

>>If no drivers need to be installed, what are the best alias names to be used in the conf file? bcm5700 or e1000 ???

The alias names for bcm5700 and e1000 are as you have written them....bcm5700 and e1000.
These are the strings you would use for the respective drivers in modprobe.conf.

>>Also, a little surprising thing is mii-tool reports network cards speed as 10 Mbps but ethtool reports 1000 Mbps

Can you post the exact commands along with the output generated please ?

>>Should we need to turn ON auto negotiation on the Server farm switches? ( We have Cisco 3750s)

Since you are using 1000 Mbps cards, it is always recommended that you enable autonegotiation. Some drivers will not operate at this speed unless auto has been enabled. The driver standard generally calls for autonegotiation to be enabled.

RE: install the bcm5700 driver. It is Broadcoms' intention to replace the bcm driver with the tg3 driver though HP continues to support the both drivers. Moving from the tg3 driver may not be the solution here.

Since you mentioned that you may not have autonegotiation set in your switches, this is more than likely the reason you are running at less than optimal speed. As mentioned above, auto is always recommended at 1000Mbps speeds. Most recent network devices default to autonegotiation. It is best to allow them to determine the most appropriate speed/duplex mode for the existing conditions.

[Matti_Kurkela]

Regarding the changing network device names:
RHEL ES/AS 4 has a new trick in network device configuration. When you configure the network interfaces during the OS installation, the installer will store the MAC addresses of each activated interface into /etc/sysconfig/network-scripts/ifcfg-ethX.

When the system is booting, the network startup script will check the MAC address of each interface detected on the system. If there is a mismatch, it will *rename the network interfaces* so that the MAC addresses will match the configuration files.

This makes sense if you consider what happens when you have several network interfaces, for example, eth0, eth1 and eth2. If you physically remove eth1 and reboot, older Linux systems would change the numbering so that the old eth2 would be now eth1. Now in addition of having the (old) eth1 physically missing, the eth2 would be mis-configured with the parameters of the (missing) eth1.

RHEL ES/AS4 will use the MAC addresses to detect that one card has gone away, rename the interfaces so that the card that was once configured as eth2 will *stay* named as eth2 and will be configured just as before removing the eth1 card.

You can remove the HWADDR line from the /etc/sysconfig/network-scripts/ifcfg-ethX files, if you wish: removing those lines restores the old behaviour.

[CA1367359]

Hello

Sorry for the delay in my reply, servers are in a separate firewall.. making it difficult to capture the settings.

Anyway here are the requested outputs (mii-tool, ethtool and modprobe.conf)

#mii-tool
eth0: 10 Mbit, full duplex, link ok
eth1: no link
eth2: no link
eth3: 10 Mbit, full duplex, link ok
#ethtool eth0
Settings for eth0:
Supported ports: [ MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: Not reported
Advertised auto-negotiation: No
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: g
Wake-on: d
Current message level: 0x000000ff (255)
Link detected: yes
#
cat /etc/modprobe.conf
alias eth0 tg3
alias eth1 tg3
alias eth2 tg3
alias eth3 tg3

As you see here, mii-tool reports them to 10 Mbps connection , while ethtool shows them as 1000 mbps.

Please advise

[Vipulinux]

Hi

Donot use the tg3 module; use bcm57xx modules and all your worries will be over.

Cheers
Vipul