interface order eth0/eth1 on HP blades

[Arnd Kohlen]

Hi,

Is there a way to tell linux which ethernet interface (on a blade) should be mapped either as eth0 or eth1?

I have a few blades detecting the first path-through module as eth1 and the second one as eth0. Other blades detect the first path-through module es eth0 and the second one as eth1. The configuration is on both blades the same. It is the same kernel, the same driver version - only the ip-addresses differ.

This is the output from dmesg (for the ethernet cards):

Broadcom NetXtreme II Gigabit Ethernet Driver bnx2 v1.5.8.1 (May 7, 2007)
ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI 16 (level, low) -> IRQ 16
eth0: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem f6000000, IRQ 16, node addr 001a4ba8bf70
ACPI: PCI Interrupt 0000:07:00.0[A] -> GSI 16 (level, low) -> IRQ 16
eth1: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit 133MHz found at mem fa000000, IRQ 16, node addr 001a4ba8bf5c
ACPI: PCI Interrupt 0000:00:1d.7[A] -> GSI 16 (level, low) -> IRQ 16

I even tried to configure the kernel, not to autoprobe the addresses for the ethernet cards but I might have a mistakes in these options:

APPEND="reserve=0x300,32 ether=0,0x300,eth1"

[Ivan Ferreira · Accepted solution]

Yes, use the /etc/sysconfig/network-scripts/ifcfg-ethX file and add:

HWADDR="interface:mac:address:"

[Arnd Kohlen]

This is the correct solution for Red Hat and maybe for SuSe. But in this case I also need to change our cluster software to make the interfaces and ip address dependable on the mac address, not on the hardware.

I wonder why some of the blades have these interfaces switched. It seems that when swithing on a blade it detects the hardware on the pci slots but the order is not allways the same. This might be a kernel or a blade hardware issue.

I know this is a special configuration ;-) but maybe someone knows how to configure the BIOS/Hardware to have the NIC on the first path-through module detected as eth0...

[Justin_99]

You might also want to have a look at the udev rules. I have seen cards get eth0,1,2.... assigned via udev.

:/etc/udev/rules.d # cat 30-net_persistent_names.rules
# This rules are autogenerated from /lib/udev/rename_netiface.
# But you can modify them, but make sure that you don't use an interface name
# twice. Also add such interface name rules only in this rules file. Otherwise
# rename_netiface will create wrong rules for new interfaces.
# It is safe to delete a rule, as long as you did not disable automatic rule
# generation. Only if all interfaces get a rule the renaming will work
# flawlessly. See also /etc/udev/rules.d/31-net_create_names.rules.
#
# Read /usr/share/doc/packages/sysconfig/README.Persistent_Interface_Names for
# further information.
#
# Use only a-z, A-Z and 0-9 for interface names!
#
SUBSYSTEM=="net", ACTION=="add", SYSFS{address}=="00:50:45:5b:e7:1a", IMPORT="/lib/udev/rename_netiface %k eth0"
SUBSYSTEM=="net", ACTION=="add", SYSFS{address}=="00:50:45:5b:e7:1b", IMPORT="/lib/udev/rename_netiface %k eth1"

[Arnd Kohlen]

Thanks Justin.

I'll keep this in mind for later steps. For now I'm still searching a possibility to get all the blades assigning the NICs in the same order.

To make the order (eth0/eth1) dependable on the mac address means a hack in our cluster software.

I'll wait for a few other suggestions. Maybe there's a setting in the bios which does a rescan of the hardware and changes the order of the onboard NICs.

Thanks again.
Arnd

[rick jones]

First, I feel your pain wrt interface naming persistens in Linux, it has bitten me _numerous_ times in the past and is a persistent source of frustration to me, coming from an HP-UX background with that sort of thing has "just worked" for decades.

Still, Linux is getting better here, and the suggestions given thusfar are the ways to go under Linux thusfar.

Can you go into a bit more detail on why your cluster software is an issue in using the different interfaces' MAC addresses to determine which should be eth0, eth1 etc?

[Arnd Kohlen]

Rick,
thanks for helping.

Well, its a cluster software wich we developed (and are still developing on it ;-) ). We have aprox. 50 DL380G3 and G4 and the interface assignment under linux depended on the number of the PCI-Slot where the NIC was built in. For example the (dual-)NIC in the PCI-Slot 1 was allways eth0, eth1. The NIC in PCI-Slot2 was eth2, eth3.

Now we have 16 blades and while starting the first blades over our cluster the first interface was eth1 and the second was eth0 (which first was confusing but we thought, ok - that's for the blades ok). This was fine for 5 Blades. The 6th and 7th blade just changed this order (it is the same Kernel, the same driver version...) and we don't see any reason.

Ok, back to your question. Our cluster software has a field in the database which defines the first, booting interface: eth1. This was for the DL380 eth0, for the blades eth1. If we make this dependable on the mac address we need to change the database layout and many parts of the cluster software.

That's why I first asked here in the forum. I'm looking for an option in the bios or something like "plug the blade out and back again to have the onboard nics reassigned" ;-). But it seems like we have to change our cluster software (which btw. is much better).

Regards
Arnd

[rick jones]

I think that what people were suggesting wasn't that your cluster software become MAC aware, but that you use your externally gained knowledge of the MAC's of the various ports to cause udev or the interface files to have the ports you are used to being eth0/eth1 actually be eth0/eth1 like your software is expecting.

If I understood your description, your cluster software depends on the interface named "eth1" being a particular interface. Udev/whateverthatotheriscalled can allow you to make that interface be called eth1 by telling it that the interface with MAC foo should be called eth1.