lacp negotiation too slow across separate PCI cards during reboot

[Tim Nelson]

Anyone have any suggestions for the below:

I have two dual ported 10Ge PCI cards with one port each in a mode4 bond.

(Server running Oracle VM 3.0.3)

during reboot, it seems that the negotion is a little slow as I get "no link" error messages when rebooting.

Once booted, the bond recovers itself without issue.

If I bond the two interfaces on the same PCI card then there are no issues.

The delay in the bond becoming ready causes the bond interface not being attached to the virutal bridge.

I can put a delay in between the ifup bond and ifup bridge but I was looking to see if anyone had any other suggestions before I add the manual delay.

[Tim Nelson]

I found a solution. (reminder, I am using Oracle Virtual Machine-XEN OS)

If you read through /etc/sysconfig/network-scripts/ifup-eth, you will see the section(around line 327) where the interface is created and brought up and then added to the bridge.  The script references a variable LINKDELAY to cause a sleep for this amount of time (e.g. LINKDELAY=30 ) .

By adding LINKDELAY=30 to the ifcfg-bond0 file, this gives ample time for the bond0 interface to come up due to slow LACP negotiation before the script attempts to add bond0 to the bridge.

I might expect this to possibly be an issue with VMWare also as the backend OS is similar. (or maybe VMWare does not have an issue or has already identified it).

the 30 second delay is just a number i picked, there are maybe 4 places in the ifup-eth script that use this as a sleep value, so 30 seconds really turns into about 2 minutes.

Will have debug turned on in the switches to see what the real delay is on the channel-group creation.  I can then tweek the delay to reduce boot time.