RHEL4 - NIC bonding - Wrong speed

[Allan Andersen_2]

Hi everybody

I'm trying to get nic bonding to work on a dl 360 g4p with bcm5700. The bonding works, but i can only get 100 Mbit from each nic.

dmesg:

eth0: Using PCI INTX interrupt
bonding: Warning: failed to get speed/duplex from eth0, speed forced to 100Mbps, duplex forced to Full.
bonding: bond0: enslaving eth0 as an active interface with an up link.
eth1: Using PCI INTX interrupt
bonding: Warning: failed to get speed/duplex from eth1, speed forced to 100Mbps, duplex forced to Full.
bonding: bond0: enslaving eth1 as an active interface with an up link.
bcm5700: eth0 NIC Link is UP, 1000 Mbps full duplex
bcm5700: eth1 NIC Link is UP, 1000 Mbps full duplex
bond0: no IPv6 routers present
eth0: no IPv6 routers present
eth1: no IPv6 routers present


cat /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
BOOTPROTO=none
MASTER=bond0
SLAVE=yes
USERCTL=no
HWADDR=00:15:60:AC:17:1A
ONBOOT=yes
TYPE=Ethernet
ETHTOOL_OPTS="speed 1000 duplex full autoneg off"

cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
BOOTPROTO=none
USERCTL=no
MASTER=bond0
SLAVE=yes
HWADDR=00:15:60:AC:17:19
ONBOOT=yes
TYPE=Ethernet
ETHTOOL_OPTS="speed 1000 duplex full autoneg off"

cat /etc/sysconfig/network-scripts/ifcfg-zbond0
DEVICE=bond0
IPADDR=10.1.36.4
NETMASK=255.255.252.0
NETWORK=10.1.36.0
GATEWAY=10.1.36.1
#BROADCAST=1.1.39.255
ONBOOT=yes
BOOTPROTO=none
USERCTL=no


ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Link detected: yes

ethtool eth1
Settings for eth1:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised auto-negotiation: Yes
Speed: 1000Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 0
Transceiver: internal
Auto-negotiation: on
Supports Wake-on: g
Wake-on: d
Link detected: yes

cat /proc/net/bonding/bond0
Ethernet Channel Bonding Driver: v2.6.1 (October 29, 2004)

Bonding Mode: load balancing (round-robin)
MII Status: up
MII Polling Interval (ms): 0
Up Delay (ms): 0
Down Delay (ms): 0

Slave Interface: eth0
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:15:60:ac:17:1a

Slave Interface: eth1
MII Status: up
Link Failure Count: 0
Permanent HW addr: 00:15:60:ac:17:19


rpm -q bcm5700
bcm5700-8.3.14a-1

[Vipulinux]

Hi
You can try setting up the speed with mii-tool.

Hope that works

Cheers

[Allan Andersen_2]

isn't that tool replaced by ethtool? And also it doesn't support gigabit

[Ivan Ferreira]

See this thread:

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=989331

[Allan Andersen_2]

I have seen that tread before. But unable to find a solution in it?

[Ivan Ferreira]

But ethtool reports

Speed: 1000Mb/s
Duplex: Full

Can you verify the speed on the switch? Do you have any output errors?

[Allan Andersen_2]

yes ethtool reports says 1 Gb. But when they are bonded dmesg says they are forced to 100Mbit

I have done a troughput test where i had 2 Gbit in the other end and i was only able to transfer 7-8 MB/s

[Allan Andersen_2]

ifconfig
bond0 Link encap:Ethernet HWaddr 00:15:60:AC:17:1A
inet addr:10.1.36.4 Bcast:10.1.39.255 Mask:255.255.252.0
inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link
UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1
RX packets:317171 errors:0 dropped:0 overruns:0 frame:0
TX packets:579594 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:22648663 (21.5 MiB) TX bytes:861301188 (821.4 MiB)

eth0 Link encap:Ethernet HWaddr 00:15:60:AC:17:1A
inet6 addr: fe80::215:60ff:feac:171a/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:158571 errors:0 dropped:0 overruns:0 frame:0
TX packets:289795 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:11320657 (10.7 MiB) TX bytes:431420250 (411.4 MiB)
Interrupt:209 Memory:fdf70000-fdf80000

eth1 Link encap:Ethernet HWaddr 00:15:60:AC:17:1A
inet6 addr: fe80::215:60ff:feac:171a/64 Scope:Link
UP BROADCAST RUNNING SLAVE MULTICAST MTU:1500 Metric:1
RX packets:158600 errors:0 dropped:0 overruns:0 frame:0
TX packets:289799 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:11328006 (10.8 MiB) TX bytes:429880938 (409.9 MiB)
Interrupt:217 Memory:fdf60000-fdf70000

[Vitaly Karasik_1]

It seems to be some bug in linux bonding code. In bond_main.c I see a code which should get ethX speed/duplex status using ethtool API, but default value is 100MB.
So for some reason (or bugs in bonding code, or some interoperability issues with bcm5700 driver) bonding program cannot detect that eth0 work 1GB...
Do you use the latest RHEL4 update?

[Vitaly Karasik_1]

It seems to be some bug in linux bonding code. In bond_main.c I see a code which should get ethX speed/duplex status using ethtool API, but default value is 100MB.
So for some reason (or bugs in bonding code, or some interoperability issues with bcm5700 driver) bonding program cannot detect that eth0 work 1GB...
Do you use the latest RHEL4 update?

here http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=977328 we have similar thread.

[Steven E. Protter]

/etc/modprobe.conf

??

alias bond0 bonding
options bond0 miimon=200 mode=active-backup
alias scsi_hostadapter mptbase
alias scsi_hostadapter1 mptscsih
alias scsi_hostadapter2 ata_piix
alias usb-controller ehci-hcd
alias usb-controller1 uhci-hcd
alias eth1 tg3


What does yours say?

Are you using tg3 or bcm5700 drivers?

SEP-

[Allan Andersen_2]

i'm so sorry but i have figured out the problem and it had nothing to do with the error in kernel log.


The server is on another vlan than the client testing from and therefore had to go trough another city(Max 155 MBit) I have tested with a server connected to the same vlan at 1 Gbit and was able to transfer with 40 MB/s so i assume it's working and the error in the kernel log just got me to look in the worng place.

[Steven E. Protter]

I glad you found your problem and corrected it.

I urge you to assign points to other peoples responses in this question thread.

It encourages further response.

If you feel the response had no value, such as my two, assign zero.

:=)

SEP

[Allan Andersen_2]

i'm so sorry but i have figured out the problem and it had nothing to do with the error in kernel log.


The server is on another vlan than the client testing from and therefore had to go trough another city(Max 155 MBit) I have tested with a server connected to the same vlan at 1 Gbit and was able to transfer with 40 MB/s so i assume it's working and the error in the kernel log just got me to look in the worng place