RX errors on Ethernet Adapter

[Matthias Kretschmer]

Hello,
I have problems with an Ethernet Adapter in my DL380 G7 with RHEL 5.3.
We have four NICs. Two on board and two via PCIe.
The both Nics on the PCIe cards have a lot of RX errors. As soon as I bind these NICs into a bonding the behavior is very strange.
The bond is good as long as the active NIC is one of the on-board NICs. As soon as there is a failover to a PCIe NIC I can ping the bond IP but I can not connect via SSH.

I have a duplicate server next to that, that has the same configuration, but is running fine. I switched the hard disks of the two servers to test if this is a configuration problem in my bonding, but the problem stays on the same server. Therefore I believe it is a problem on the hardware itself.

I do not know where to start and how to investigate that.
Here some outputs of my server (I removed eth2 from bonding for testing).

# ifconfig eth2
eth2 Link encap:Ethernet HWaddr 1C:C1:DE:18:3E:8E
inet addr:10.10.145.65 Bcast:10.10.255.255 Mask:255.255.128.0
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:71510 errors:6423 dropped:0 overruns:0 frame:6423
TX packets:5 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6034633 (5.7 MiB) TX bytes:320 (320.0 b)
Interrupt:186 Memory:f8000000-f8012100

# ethtool eth2
Settings for eth2:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: g
Link detected: yes

# mii-tool eth2
eth2: negotiated 100baseTx-FD flow-control, link ok

Would be great to get some help here.
Thank you in advance.

Matthias

[Matthias Kretschmer]

I am sorry.
I have to correct myself.
Of cause, the DL380 G7 has four on-board NICs and no PCIe Card with NICs.
This means it is even more strange, because NIC 1 and 2 are running fine, but 3 and 4 have a lot of RX Errors.

BR
Matthias

[Steven Schweda]

Have you tried changing any hardware (cables,
switch ports, ...)?

> Settings for eth2:

And what are the settings at the other end of
the cable?

[rick jones]

Strictly speaking, the minimum version of RHEL5 supported on the DL380 G7 is 5.4:

http://h18000.www1.hp.com/products/servers/linux/supportmatrix/rhel/exceptions/rhel-exceptions.html

You might get more information about the frame errors if you look at ethtool -S .

Assuming that there are cables connected to all four of the LOM ports, and they go to the same switch, and indeed only two of them show errors, you might consider "rotating" the cables one port and see if the errors stay with the port, or move with the cable. If they move with the cable, then repeat that on the switch, and see if they move with the cable, or stay with the port.