Pause Frame Flood problem question

[chuckk281]

Sukru was looking to solve a customer problem:

************

Hi All,

We have recently experienced a Pause Frame Flood problem. The VC modules have shut down all the downlink ports of a Blade server for preventing the Pause Frame Flood creates further problem in the VC modules. The support people have helped the customer to reset the server and the associated ports and the problem has been cleared now. As an ASM I need to explain the cause of the problem and at least give some recommendation to the customer to prevent the problem ocurring in the future.

As far as I understand such a problem may be caused by the following two reasons:

1. There may be a bug in the NIC firmware and/or NIC driver of the Blade server causing the server sends excessive pause frames to the downlink ports of the VC.
2. There is really a situtaion in the network that caused the Blade server sending pause frames frequently (or excessively).

We have confirmed that the NIC firmware/driver of the Balde Server is the latest:

Server: BL490c G6, NIC: Broadcom NetXtreme II BCM57711E/NC532i 10 Gig Eth.

OS: VMware ESXi 5.5

NIC Fw/Driver: 2.710.39.v55.2 / bc 6.2.28 phy baa0.105

Although it may still be a firmware/driver bug I cannot recommend an upgrade since it is already the latest.

I have thought that disabling the Ethernet Flow Control on the server may prevent such a problem in the future because the Pause Frames are sent when Ethernet Flow Control is enabled. On the other hand we have another more sophisticated  flow control mechanism on TCP level and disabling the flow control on Ethernet level may not create a problem.

I would like to ask you about your opinion on disabling Ethernet Flow Control on the server NIC for preventing such a problem causing the VC shutting down the server ports.

I appreciate your recppmmendations,

************

Input from Will:

************

You might want to check the customer’s firmware version for Virtual Connect.  I believe it was version 4.30 that addressed/resolved some pause frame issues.  If this is the case, verify they are on the latest ESXi 5.5 driver as well prior to updating.  Hope this helps.

**********

Other comments?

[Susan Lewis]

For what it's worth, we saw a similar issue on Integrity systems with that same Broadcom device and a Procurve switch.

We were told it was fixed with a new Broadcom driver last fall (2014). It's not clear to me if it was a run-time driver or a boot-driver (UEFI).  The problem report does not show the new driver version, unfortunately.

Good luck,

Sue L

p.s. I am an HP employee.

[bmoremike]

We had the same issue with pause frames.  The NICs in the G6 can not handle a mtu larger than 4k.  Once we lowered the MTU the port flood issue went away.