BladeSystem Management Software

NIC enumeration issues

 
John Cagle
Frequent Advisor

NIC enumeration issues

This topic is to discuss problems with various Linux kernels and the way they number the NICs (eth0, eth1, eth2, etc.).
5 REPLIES 5
John Cagle
Frequent Advisor

NIC enumeration issues

A customer has reported an issue with SystemImager on the BL480c, where NIC1 is enabled for PXE by default, and SystemImager boots, but then it discovers NIC3 as "eth0". This breaks the SystemImager installation because (1) NIC3's MAC address is different from NIC1, and (2) NIC3 may not be connected to the PXE network. Here is the solution I proposed: This is not a simple issue to explain, but, in the end, it is a Linux kernel issue. Server hardware designers have always relied on PCI (Bus/Device/Function) as the mechanism to order the NICs. Current Linux kernels use various methods to enumerate the NICs, some of them breadth-first and some depth-first. Neither method is “correct� according to the way modern servers are designed (at least those with multiple NICs). RHEL4 Update 5 will contain a workaround, and you can get a test kernel here: http://people.redhat.com/~jbaron/rhel4/ The workaround will discover the NICs in the proper order (NIC1, NIC2, NIC3, NIC4) by sorting them using their PCI Bus, Device, and Function numbers. So basically, you have three options – 1) Enable PXE only on NIC3, then this will match “eth0� as enumerated by the SystemImager kernel. 2) Modify the SystemImager kernel to name NIC1 as “eth0�, perhaps using a script that can detect the PXE MAC address and then use that to dynamically determine which NIC is NIC1. 3) Use RHEL4 Update 5’s kernel for SystemImager.
Frank Sorenson
New Member

NIC enumeration issues

We saw this issue as well, essentially with all HP hardware (and only HP hardware so far). We've found that using "ksdevice=MAC_ADDR" on the kernel boot line solves the problem for us. For example: KERNEL vmlinuz-rhel5-64 APPEND initrd=initrd-rhel5-64.img ks=nfs:w.x.y.z:/vol/ks.cfg ksdevice=00:1a:4b:dd:cd:d8 ip=w.x.y.z netmask=255.255.255.128 gateway=w.x.y.z dns=w.x.y.z Not sure if this will help SystemImager, but it works well for standard kickstart.
John Cagle
Frequent Advisor

NIC enumeration issues

Thanks for the tip! I'm sure it will help someone else. Also, RHEL4 Update 5 has the fixes for HP BladeSystem products, so "eth0" should now correspond to NIC1 (the default PXE NIC) as long as you use an Update 5 kernel. By the way, this issue is not HP-specific. It happens on other brands too.
violetlight
Occasional Advisor

NIC enumeration issues

Has anyone seen a fix for SLES10 or SLES10SP1? I'll chase this with Novell too. Thanks Simon.
John Cagle
Frequent Advisor

NIC enumeration issues

We shared this patch with Novell, but they chose to not include it in their kernels. If you feel that it's necessary in SLES kernels, please send your feedback to Novell.