Eth0 and Eth1 Looses Information after Reboot

[ramizkhan]

Problem:
=========
Eth0 and Eth1 is bonded and works fine
but when i reboot the OS eth0 and eth1 cant be up when i run command "ifup bond0" , it gives me following message.



netxen_nic device eth0 doesnt seem to be present, delaying intialization


netxen_nic device eth1 doesnt seem to be present, delaying intialization


Server: HP 580 Proliant

OS information:
==============

[root@at1ohypdb101 admin]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.3 (Tikanga)

[root@at1ohypdb101 admin]# uname -r
2.6.18-128.el5

Network Card :
=============

HP NC522SFP Dual Port 10GbE Gigabit Server Adapter


I am using above card , it works perfectly fine and I have bonded but when I reboot , eth0 and eth1 looses this driver and also the RPMS I have installed , they are gone too.

Any help would be highly appreciated.

Thanks,

[Elmar P. Kolkman]

Check out your ocnfiguration files...
You might be missing the modules, needed for using the cards. Perhaps some misconfiguration in the modprobe.conf or something like that.
Check out the lspci output (to make sure the card is still visible).
Also a ifconfig -a might help to find the cause.
I think for this card you need the netxen_nic module.

[ramizkhan]

This is my modprobe.conf file

==============
alias bond0 bonding
alias eth0 bnx2
alias eth1 bnx2
alias eth2 bnx2
alias eth3 bnx2
alias scsi_hostadapter cciss
alias scsi_hostadapter1 qla2xxx
alias scsi_hostadapter2 usb-storage
###BEGINPP
include /etc/modprobe.conf.pp
###ENDPP
alias eth4 e1000e
==================

Also, lscpsi out shows following


13:00.0 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10 Gigabit Server Adapter (rev 42)

13:00.1 Ethernet controller: NetXen Incorporated NX3031 Multifunction 1/10 Gigabit Server Adapter (rev 42)
[root@at1osoadb101 ~]#


When you say modules ?
What Exactly do you mean by modules ?
I did install the driver of this card , and when I can see that driver ., when i do

rpm -qa | grep hp

thanks

[ramizkhan]

Sorry please disregard my above message

My modprobe.conf is following

======
[root@at1osoadb101 ~]# cat /etc/modprobe.conf
alias bond0 bonding
alias eth0 netxen_nic
alias eth1 netxen_nic
alias eth2 bnx2
alias eth3 bnx2
alias scsi_hostadapter cciss
alias scsi_hostadapter1 ata_piix
alias scsi_hostadapter2 qla2xxx
alias scsi_hostadapter3 usb-storage
###BEGINPP
include /etc/modprobe.conf.pp
###ENDPP
==========

[Steven E. Protter]

Shalom,

cat /etc/redhat-release
uname -a
# please post.

I'm not sure bonding of 10BbE interfaces is even supported at this point.

SEP

[Modris Bremze]

Also, the output of
/etc/sysconfig/network-scripts/ifcfg-bond0
/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/network-scripts/ifcfg-eth1
could be useful.

What exactly did you mean by "the RPMS I have installed , they are gone too."?

[Gerardo Arceri]

ramizkhan:
I've been through the same problem, and the solution is to disable "udev hotplug" of the drivers so they are loaded at the moment the network interfaces are started and not previously by udev.
Go ahead and create a file called /etc/modprobe.d/networkfix containing
##### DO NOT AUTOLOAD THESE MODULES
blacklist bnx2x
blacklist e1000e
#####

Reboot and you should get the interfaces working properly, btw, i couldn't help but notice that you are bonding both eth0 and eth1 together, which if redundancy is your objective is a very bad idea (TM), i would use one iface on the onboard bnx2x card and the other on the dual e1000e, so if the onboard card happens to fail it will not bring down the whole bonded iface.

Regard and i hope this information was helpful, if so, please don't forget to assign points.

[ramizkhan]

Thanks Gerardo Arceri,

Before creating the file you requested above, I went to the following directory and find this ..

[root@at1osoadb102 netxen]# cd /etc/modprobe.d/
[root@at1osoadb102 modprobe.d]# ls -l
total 44
-rw-r--r-- 1 root root 831 Jun 30 11:46 blacklist
-rw-r--r-- 1 root root 833 Jan 21 2009 blacklist-compat
-rw-r--r-- 1 root root 83 Jan 21 2009 blacklist-firewire
-rw-r--r-- 1 root root 810 Jun 30 11:46 blacklist.saved
-rw-r--r-- 1 root root 6111 Jan 21 2009 modprobe.conf.dist


and when I view the "blacklist" file and i found following entry in it . Now do you think because of this following entry I am loosing the driver ? Should I go ahead remove this entry ?

========
blacklist netxen_nic
=======



Also, the /etc/modprobe.conf looks following
======
alias bond0 bonding
alias eth0 nx_xport
alias eth1 nx_xport
alias eth2 bnx2
alias eth3 bnx2
alias scsi_hostadapter cciss
alias scsi_hostadapter1 ata_piix
alias scsi_hostadapter2 qla2xxx
alias scsi_hostadapter3 usb-storage
###BEGINPP
include /etc/modprobe.conf.pp
###ENDPP
install netxen_nic /bin/true
install nx_xport /sbin/modprobe nx_nic || /sbin/modprobe nx_xport
~
================







Also, the suggestion you gave about bonding 10gb port with Broadcom on board nic, My question is that Broadcom card is 1gb and Hp Card is 10gb , will that work ?


Also, one more thing to look here , I just rebooted the server and it come up fine and i can putty the server and bonding is there now.. Wiered rite ? But after 3 or 4 reboot it will loose everything , I am pasting the log that i got yesterday when I was not able to putty it. It throw following messages ...

===/var/log/messages/ from Yesterday=====

Jun 30 06:27:23 at1osoadb102 kernel: unm_init_module: Remove the nx_nic driver first
Jun 30 06:27:23 at1osoadb102 modprobe: FATAL: Error inserting nx_xport (/lib/modules/2.6.18-128.el5/extra/hp-nx_nic/nx_xport.ko): Operation not permitted
Jun 30 06:27:23 at1osoadb102 kernel: unm_init_module: Remove the nx_nic driver first
Jun 30 06:27:23 at1osoadb102 modprobe: FATAL: Error inserting nx_xport (/lib/modules/2.6.18-128.el5/extra/hp-nx_nic/nx_xport.ko): Operation not permitted
Jun 30 06:27:23 at1osoadb102 kernel: unm_init_module: Remove the nx_nic driver first
Jun 30 06:27:23 at1osoadb102 modprobe: FATAL: Error inserting nx_xport (/lib/modules/2.6.18-128.el5/extra/hp-nx_nic/nx_xport.ko): Operation not permitted
Jun 30 06:27:23 at1osoadb102 kernel: unm_init_module: Remove the nx_nic driver first
Jun 30 06:27:23 at1osoadb102 modprobe: FATAL: Error inserting nx_xport (/lib/modules/2.6.18-128.el5/extra/hp-nx_nic/nx_xport.ko): Operation not permitted
Jun 30 06:27:23 at1osoadb102 kernel: unm_init_module: Remove the nx_nic driver first
Jun 30 06:27:23 at1osoadb102 modprobe: FATAL: Error inserting nx_xport (/lib/modules/2.6.18-128.el5/extra/hp-nx_nic/nx_xport.ko): Operation not permitted
Jun 30 06:27:23 at1osoadb102 kernel: unm_init_module: Remove the nx_nic driver first
Jun 30 06:27:23 at1osoadb102 modprobe: FATAL: Error inserting nx_xport (/lib/modules/2.6.18-128.el5/extra/hp-nx_nic/nx_xport.ko): Operation not permitted
Jun 30 06:27:23 at1osoadb102 kernel: unm_init_module: Remove the nx_nic driver first
Jun 30 06:27:23 at1osoadb102 modprobe: FATAL: Error inserting nx_xport (/lib/modules/2.6.18-128.el5/extra/hp-nx_nic/nx_xport.ko): Operation not permitted
Jun 30 06:27:23 at1osoadb102 kernel: unm_init_module: Remove the nx_nic driver first
Jun 30 06:27:23 at1osoadb102 modprobe: FATAL: Error inserting nx_xport (/lib/modules/2.6.18-128.el5/extra/hp-nx_nic/nx_xport.ko): Operation not permitted

Jun 30 07:22:06 at1osoadb102 gconfd (root-8067): Resolved address "xml:readwrite:/root/.gconf" to a writable configuration source at position 0
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: -+ //etc/modprobe.conf eth0 alias nx_nic
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: -+ //etc/modprobe.conf eth1 alias nx_nic
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: -+ //etc/modprobe.conf eth2 alias bnx2
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: -+ //etc/modprobe.conf eth3 alias bnx2
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: -+ //etc/modprobe.conf bond0 alias bonding
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: chmod 0644 //etc/sysconfig/networking/devices/ifcfg-eth2
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: chmod 0644 //etc/sysconfig/networking/devices/ifcfg-bond0
Jun 30 07:23:04 at1osoadb102 nm-system-settings: ifcfg-rh: updating /etc/sysconfig/network-scripts/ifcfg-bond0
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: chmod 0644 //etc/sysconfig/networking/devices/ifcfg-eth3
Jun 30 07:23:04 at1osoadb102 nm-system-settings: ifcfg-rh: updating /etc/sysconfig/network-scripts/ifcfg-eth3
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: rm //etc/sysconfig/networking/devices/ifcfg-eth0
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: rm //etc/sysconfig/network-scripts//ifcfg-eth0
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: rm //etc/sysconfig/networking/devices/ifcfg-eth1
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: rm //etc/sysconfig/network-scripts//ifcfg-eth1
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: rm //etc/sysconfig/networking/profiles/default/ifcfg-eth0
Jun 30 07:23:04 at1osoadb102 system-config-network[8208]: rm //etc/sysconfig/networking/profiles/default/ifcfg-eth1
Jun 30 07:23:05 at1osoadb102 nm-system-settings: ifcfg-rh: parsing /etc/sysconfig/network-scripts/ifcfg-eth0 ...
Jun 30 07:23:05 at1osoadb102 nm-system-settings: ifcfg-rh: error: Couldn't parse file '/etc/sysconfig/network-scripts/ifcfg-eth0'
Jun 30 07:23:05 at1osoadb102 nm-system-settings: ifcfg-rh: parsing /etc/sysconfig/network-scripts/ifcfg-eth1 ...
Jun 30 07:23:05 at1osoadb102 nm-system-settings: ifcfg-rh: error: Couldn't parse file '/etc/sysconfig/network-scripts/ifcfg-eth1'
Jun 30 07:28:44 at1osoadb102 system-config-network[8276]: -+ //etc/modprobe.conf eth0 alias nx_nic
Jun 30 07:28:44 at1osoadb102 system-config-network[8276]: -+ //etc/modprobe.conf eth1 alias nx_nic
Jun 30 07:28:44 at1osoadb102 system-config-network[8276]: -+ //etc/modprobe.conf eth2 alias bnx2
Jun 30 07:28:44 at1osoadb102 system-config-network[8276]: -+ //etc/modprobe.conf eth3 alias bnx2
Jun 30 07:28:44 at1osoadb102 system-config-network[8276]: -+ //etc/modprobe.conf bond0 alias bonding

[ramizkhan]

Steven E. Protter

Yes it is supported ,

[root@at1osoadb102 modprobe.d]# uname -r
2.6.18-128.el5
[root@at1osoadb102 modprobe.d]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.3 (Tikanga)
[root@at1osoadb102 modprobe.d]# uname -a
Linux at1osoadb102.onehreem.com 2.6.18-128.el5 #1 SMP Wed Jan 21 08:45:05 EST 2009 x86_64 x86_64 x86_64 GNU/Linux




you can take a look this is the card

http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=us&prodNameId=3913538&prodTypeId=329290&prodSeriesId=3913537&swLang=13&taskId=135&swEnvOID=4004

[ramizkhan]

I am posting the ifconfig -a
because Server is up now, but i am pretty sure If i reboot 2 or 3 times netowrk is gona go and it isgoing to loose the driver

===
inet addr:10.92.200.10 Bcast:10.92.200.255 Mask:255.255.255.0
inet6 addr: fe80::f6ce:46ff:feaf:6e00/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:18511 errors:0 dropped:0 overruns:0 frame:0
TX packets:11914 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1431792 (1.3 MiB) TX bytes:23096994 (22.0 MiB)

eth0 Link encap:Ethernet HWaddr F4:CE:46:AF:6E:00
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:17983 errors:0 dropped:0 overruns:0 frame:0
TX packets:11709 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1378030 (1.3 MiB) TX bytes:23071650 (22.0 MiB)
Interrupt:75

eth1 Link encap:Ethernet HWaddr F4:CE:46:AF:6E:00
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:528 errors:0 dropped:0 overruns:0 frame:0
TX packets:207 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:53762 (52.5 KiB) TX bytes:25668 (25.0 KiB)
Interrupt:139

eth2 Link encap:Ethernet HWaddr 18:A9:05:59:2E:66
inet6 addr: fe80::1aa9:5ff:fe59:2e66/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:5434 (5.3 KiB)
Interrupt:169 Memory:dc000000-dc012100

eth3 Link encap:Ethernet HWaddr 18:A9:05:59:2E:68
inet addr:192.168.1.8 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::1aa9:5ff:fe59:2e68/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:56 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 b) TX bytes:9768 (9.5 KiB)
Interrupt:177 Memory:de000000-de012100

=====


root@at1osoadb102 netxen]# /opt/netxen/nxudiag -i eth0
NETXEN Interface is eth0
Interface eth0 is UP.
************* Board Info *************
Board Ser# : CB01BK0937
Chip Rev : B2
Board Type : 0x26 (HP NC522SFP Dual Port 10GbE Server Adapter)
Core clock : 330 MHz
Mem clock : 300 MHz
Crystal freq : 20 MHz
SRE Mode : LEGACY
Firmware in : LEGACY mode
DDR size : 256 MB, ECC Enabled
QDR size : 64 MB, ECC Enabled
Peg ICACHE : OK, Enabled
Peg DCACHE : OK, Enabled
Firmware ver.: 4.0.526
Driver ver. : 4.0.520
MAC Addr 0 : F4:CE:46:AF:6E:00
MAC Addr 1 : F4:CE:46:AF:6E:04
**************************************
Junction Temperature = 38 degrees C , Status = 'NORMAL'
[root@at1osoadb102 netxen]#