Virtual Connect FlexFabric interconnect modules and Ethernet Flow Control

[chuckk281]

Chris had a question regarding VC uplink flow control:

*******************************

There is a recommendation to use Ethernet Flow Control on the switches when using the iSCSI P4000 storage nodes:

IP storage networks are unique in the amount of sustained

bandwidth that is required to maintain adequate performance

levels under heavy workloads. Gigabit Ethernet Flow Control

(802.3x) technology should be enabled on the switch to

eliminate receive and/or transmit buffer cache pressure. The

storage nodes should also be set to have flow control enabled.

Note: some switch manufacturers do not recommend

configuring Flow Control when using Jumbo Frames, or vice

versa. Consult the switch manufacturer documentation for

guidance on this issue. LeftHand Networks recommends

implementing Flow Control over Jumbo Frames for optimal

performance.

Flow control is required when using DSM/MPIO.

Question:

Is Ethernet Flow Control available on ports of FlexFabric interconnect modules that are used for uplinks?

********************

From Lionel:

Flow Control is by default only enabled on the VC downlink ports, to enable Flow control on the uplinks as well, just enter the following command under the VC CLI (be careful it’s traffic disruptive!)

#set advanced-networking FlowControl=on

From Mark:

You can switch back and forth with different flow control settings using the advanced VCM CLI command.  It will be a link bounce each time you change it (potential outage). 

Just to reiterate, when we say we turn it “ON”, it’s just setting autoneg to advertise we support flow control for all ports that do autoneg (uplinks, stackinglinks, server links).  The end result is still dependent on the link partner config.  For ports that don’t support autoneg (e.g. 10G faceplate technologies), we do just turn on flow control since there’s no autoneg.  Same thing for “OFF”.  Turning off means setting to *not* advertise support for flow control for autoneg ports, or turn off completely for non-autoneg ports.

This applies regardless of what mode the FlowControl state is.

“ON” - all ports will advertise support for flow control (if autoneg), or flowcontrol turned on (non-autoneg).

“OFF” - all ports will advertise *no* support for flow control (if autoneg), or flowcontrol turned off (non-autoneg).

“Auto” - all uplink/stacking links will behave like “OFF”, and all server links behave like “ON”.

******************

Any comments or best practices on your experienes with flow control?