HPE Community read-only access December 15, 2018
This is a maintenance upgrade. You will be able to read articles and posts, but not post or reply.
Hours:
Dec 15, 4:00 am to 10:00 am UTC
Dec 14, 10:00 pm CST to Dec 15, 4:00 am CST
Dec 14, 8:00 pm PST to Dec 15, 2:00 am PST
BladeSystem - General
cancel
Showing results for 
Search instead for 
Did you mean: 

Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

 
SOLVED
Go to solution
Nima Gh.
Regular Advisor

Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

We have a c7000 blade system.
We are sending packets on 5Mpps from external server (with ixgbe card) through UPlink X2 to a server blade with BroadCom 10G LOM through internal port d15.
We saw the received amount of 1.8Mpps!! then we monitored the UPlink X2 and d15 internal port.
The IN traffic to X2 was 5Mpps but the OUT traffic of d15 port was 1.8 Mpps!!!
It seems that flex-10 itself drops the packets!!!
Is it possible???
17 REPLIES
Frank Gessner
Frequent Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

Hi Nima,

whats the speed of the assigned network in the pofile of the blade where you are sending the packets? >= 5MB ?
Which Version of bnx2x-driver are you using.
>= 1.52-14 ?
Which OS?

regards, Frank
rick jones
Honored Contributor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

What was the CPU uiltization of each core in the blade? It is entirely possible that you were saturating one of the cores taking interrupts from the NC532i. Particularly if stream of packets was considered a single "flow" by the NIC (ie if they are all to/from the same IP address) and so were not spread-out across cores.
there is no rest for the wicked yet the virtuous have no pillows
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

Is it possible for Flex-10 to negotiate with the NIC Cards and perform kind of an overflow control? Is there any way to disable this overflow mechanism for VC Flex-10?
rick jones
Honored Contributor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

10GbE standards offer the prospect of flow-control via things called pause frames. I believe it is possible for a LOM to send pause frames to the Flex-10.

While you are checking the per-core CPU utilization on the blade, you can also look at the ethtool -S statistics and see if any of them talk about sending pause frames.
there is no rest for the wicked yet the virtuous have no pillows
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

This is the point: It seems that the Flow Control is already enabled and we want to disable it!
I clarified my purpose here:
http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1447807

Anyway, it would be great if there could be anyway to turn off the flow control of either the FlexNIC or Flex-10 VC.
HEM_2
Honored Contributor
Solution

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

you can turn off flow control globally in VC after firmware 2.30 or later.

To turn it off log in to the CLI and issue the following command:

set advanced-networking FlowControl=off

*** IMPORTANT - you need to do this in a scheduled downtime or be aware that when you make this setting the VC downlink ports will be temporarily disconnected to affect the change.
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

Thank you very much!
And a point! what is PacketBufferOverallocationRatio Parameter?
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

I even changed the over allocation parameter but there is no change!
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

A point! it seems that the 2mpps is the maximum packet rate of the bnx2x!

Are embedded NICs of Proliant G6 Blade Servers assumed as ports of a single dual-port ethernet card or two different single-port cards?
rick jones
Honored Contributor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

2mpps might be the pps limit for the HW driven by the bnx2x driver, but it is not necessarily the limit of the bnx2x driver itself. On the half-height blades, there is a single, dual-port LOM. The full-height "i2" Integrity blades have a pair of dual-port LOMs. I don't recall how many are on the full-height ProLiant blades.

I am not sure that all G6 blades have the same NC532m (driven by the BCM57711E chip) LOM. Some may have a different chip for their LOM. Best check the quickspecs etc of each blade.

Also, I am informed the PPS limit of the BCM57711E is 2Mpps in each direction.
there is no rest for the wicked yet the virtuous have no pillows
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

Thanks for your reply! As I got informed, This is the Limit of Firmware! Is there a single firmware for both NICs or there are separate firmwares?
rick jones
Honored Contributor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

The limit applies to the entire chip. As far as the *NIC* limits go, it does not matter whether one is using one port or both.
there is no rest for the wicked yet the virtuous have no pillows
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

OMG!!! Then, who enforces these limits? and Where? Is this limit is hard-wired or is matter of firmware? Ann, how could we override these limits?
rick jones
Honored Contributor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

The limit is as fast as the chip can go. It isn't "enforced" per se.

What is the end-goal of this testing? What sort of task do you wish to perform with the system?
there is no rest for the wicked yet the virtuous have no pillows
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

You can consider it as a load balancer with input rate of (3mpps,10Gbps).
rick jones
Honored Contributor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

If you do indeed need a 3Mpps input rate, and presumably a 3Mpps output rate to the "back ends" then I would suggest getting in touch with your HP sales rep and/or reseller and see about getting some of the of the 10GbE Mezz cards to try-out. While I suspect it is probably clear from the name, keep in mind that the NC532m uses the same BCM57711E chip as the NC532i LOM. Were I not one of the cobbler's children (*) :) I would be inclined to try an NC550m or NC542m.


(*) from the old saying "The cobbler's children have no shoes."

BTW, just to get the number out there, "link-rate" PPS for minimum sized frames is something like 14.7 Mpps for 10GbE. Which then leaves me wondering, so I will ask, how do you know your input pps rate will be limited to 3Mpps?
there is no rest for the wicked yet the virtuous have no pillows
Nima Gh.
Regular Advisor

Re: Flex-10 VC Drops Packets @ 5 Mega Packet Per Second

Thanks, Then I have to give up the on-board FlexNIC option and look for a mezzanine card!