Eth0 and Eth1 Looses Information after Reboot

[Matti_Kurkela]

You might want to use these commands to gather information now as the system is working OK:

ethtool -i eth0
ethtool -i eth1
lsmod | grep -e netxen_ -e nx_

Save the output of these commands to a file, e.g. "works.txt".

If the problem reappears, run the commands again, and save the results to another file, e.g. "fails.txt".

Comparing these two files might give a lot of clues about what is going wrong. The appropriate parts of the "dmesg" command output and /var/log/messages in the failing case wouldn't hurt either...

Based on what I've seen so far, I might *guess* the following:

- The driver for the 10GbE network adapter is apparently built up from several kernel modules, not just one. This is common for more complex hardware.

- Your system seems to have two sets of drivers for your 10GbE network adapter: the netxen_* set of modules, and the nx_* set. If the system tries to mix up these sets modules, it will probably fail: the system should be loading modules exclusively from either the netxen_* set or the nx_* set only.

As far as I can determine, the netxen_* set is the standard version included in the RHEL 5 distribution, and the nx_* set is provided by the HP driver RPM.

- The automatic configuration tools (like kudzu in RHEL) might have a built-in preference for the drivers included in the RHEL distribution, even though the HP-provided drivers might be better.

The "preference" might also be an accidental effect, caused by the loading order of things:
- When you install an updated kernel RPM, the modules in the HP driver RPM will need to be recompiled to match the updated kernel. Fortunately, the driver RPM probably includes a script that will do this automatically as necessary while the system is booting. But...

- If kudzu runs in the boot-up sequence *before* the module-recompilation script, it might "think" that your current NIC driver configuration is wrong (because the correct set of NIC drivers has not been recompiled yet), and start adjusting it... causing the configuration to break.

- Once the modules have been successfully recompiled and the sysadmin has fixed the configuration, the system will again be able to reboot without issues... until the next kernel upgrade happens.

Kudzu tries to add a bit of Artificial Intelligence to the hardware configuration of RHEL, but I've found it sometimes turns into Artificial Stupidity instead :)

If you want to use the HP-provided drivers, you may have to disable kudzu to stop it from changing the configuration on its own:

chkconfig kudzu off

A more appropriate fix might be to tweak the start-up order of kudzu vs. the HP RPM recompilation script, but I would want to know more about the situation before doing that.

MK

[Ishwar_1]

To retain the Same Ip Address after Reboot their are 2 possible Way in which we can achieve this.

1> DHCP server should have an entry of Server MAC address binded with static IP

2> You should make static entry of the IP Address in the below file
/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/network-scripts/ifcfg-eth1
Or
You can use command neat-tui for text mode interface.

Regards
Ishwar

[Elmar P. Kolkman]

You are missing the bonding lines in your modprobe.conf...
At least, we needed them to make sure bonding was working, so I think your system has the same problem.

The lines we inserted:

alias bond0 bonding
alias bond1 bonding
alias bond2 bonding
alias bond3 bonding
options bonding miimon=100 max_bonds=4
options bond0 miimon=100 mode=4

The last one was needed to have that bond working in an active/active setup, in combination with the Cisco setup on the switches. Your mode can be different.

As for the blacklist: that has to do with the Proliant Support Pack containing HP's version of the drivers and RedHat linux delivering its own version. Try to find out which version is newest and use that one.

And the bonding with this card works fine, as long as it is not in the wrong place in some HP servers. We had the card working until it got way to hot and shut itself down. We needed to upgrade the firmware of the card and change some BIOS setting. This happened on both the servers we had with this card at that time.

[ramizkhan]

Thanks Matti Kurkela and Gerardo Arceri .

I have sole the issue , here what i did.

turn of the kudzu
remove the driver and installed the updated 5.26 version from HP site, because RHEL call it NETXEN_NIC and HP calls it NX_XPORT so they both were getting conflicted at OS reboot and that is what causing the issue.

also ethtool -i also verifies that.

I also created the network fix file with same contentc what GERARDO said above.

Also in blacklist file , i have seen
blacklist netxen_nic too which we need so this is the proper entry.

Me and other Linux Admin totally agees wit what MATTI said above.

I really appreciate your help MATTI.


Also one more thing I would like to share here is that in RHEL 5.4 release I dont even need HP updated driver and NETXEN_NIC works perfectly fine , but we are on 5.3 so I had to update the HP driver so Probabbly RHEL 5.4 has fix in it this is whay it works on 5.4.

thanks

[Steven E. Protter]

Shalom,

Turning off Kudzu may mask a problem.

When Kudzu continually detects a change in networking, this is a sign of a potentially serious hardware problem.

I would definitely boot the server into diagnostic mode and do a full hardware diagnostic.

SEP

[Sameerna Desai]

hi
i have the similar issue on OEL 5.5 , i am using HP hp-nx_nic-4.0.534-2 driver , diabled kudzu and added nx_nic entries in modeprobe.conf file, ethernet card model is HP NC522SFP for RAC interconnect.on boards cards are also disabled.

ethtool -i eth8
driver: nx_nic
version: 4.0.534
firmware-version: 4.0.534
bus-info: 0000:21:00.0

ethtool -i eth0
driver: e1000e
version: 1.0.2-k3
firmware-version: 5.12-2
bus-info: 0000:15:00.0


cat /proc/net/bonding/bond1
Ethernet Channel Bonding Driver: v3.4.0 (October 7, 2008)

Bonding Mode: fault-tolerance (active-backup)
Primary Slave: None
Currently Active Slave: eth10
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth10
MII Status: up
Link Failure Count: 0
Permanent HW addr: d8:d3:85:a1:d0:a8

Slave Interface: eth8
MII Status: up
Link Failure Count: 0
Permanent HW addr: d8:d3:85:a0:ee:d0