topic Re: rx_fw_discards in ethtool -S in Operating System - Linux

rx_fw_discards in ethtool -S

Utsav_1 — Wed, 02 Jun 2010 20:32:47 GMT

We are seeing (what appears to be) a server hang. A bit of investigation reveals dropped packets on the switch confirmed by netstat -i output and ethtool -S eth3 output.

NIC statistics:
rx_bytes: 2698325014
rx_error_bytes: 0
tx_bytes: 165856997359
tx_error_bytes: 0
rx_ucast_packets: 35847530
rx_mcast_packets: 1668
rx_bcast_packets: 1115967
tx_ucast_packets: 109345583
tx_mcast_packets: 392908
tx_bcast_packets: 437
tx_mac_errors: 0
tx_carrier_errors: 0
rx_crc_errors: 0
rx_align_errors: 0
tx_single_collisions: 0
tx_multi_collisions: 0
tx_deferred: 0
tx_excess_collisions: 0
tx_late_collisions: 0
tx_total_collisions: 0
rx_fragments: 0
rx_jabbers: 0
rx_undersize_packets: 0
rx_oversize_packets: 0
rx_64_byte_packets: 157258
rx_65_to_127_byte_packets: 36778216
rx_128_to_255_byte_packets: 24878
rx_256_to_511_byte_packets: 4795
rx_512_to_1023_byte_packets: 18
rx_1024_to_1522_byte_packets: 0
rx_1523_to_9022_byte_packets: 0
tx_64_byte_packets: 955
tx_65_to_127_byte_packets: 446187
tx_128_to_255_byte_packets: 51731
tx_256_to_511_byte_packets: 4189
tx_512_to_1023_byte_packets: 4237
tx_1024_to_1522_byte_packets: 109231629
tx_1523_to_9022_byte_packets: 0
rx_xon_frames: 0
rx_xoff_frames: 0
tx_xon_frames: 0
tx_xoff_frames: 0
rx_mac_ctrl_frames: 0
rx_filtered_packets: 581446
rx_discards: 0
rx_fw_discards: 24168

The last line rx_fw_discards corresponds to the dropped packets shown in netstat -i.

This is a Broadcom NetXtreme II BCM5709 Gigabit Ethernet card.

Also, I see iptables reject/drop packets (as expected), but the numbers don't match.

Are rx_fw_discards harmful? Would they cause server/application/process issues/instability?

Re: rx_fw_discards in ethtool -S

Matti_Kurkela — Thu, 03 Jun 2010 12:37:57 GMT

A small number of discards should not be harmful.

Currently, the number of discards is less than 0.07 % of your incoming traffic.

Discarded packets should never cause any hardware-level issues or instability - if they do, the hardware designer has made a **major** blunder.

The same is true at the OS level.

At the application level, it depends on the network protocols used. A programmer can certainly write a network protocol that assumes everything is always perfect. Such a protocol would be extremely "fragile" and hard to use outside artificially perfect laboratory networks.

The TCP protocol contains a re-transmission feature. If the application protocol is layered on top of TCP (as most are today), then from the viewpoint of the application, a discarded and re-transmitted packet would be detectable only as a slight pause in incoming data flow.

MK

Re: rx_fw_discards in ethtool -S

michael chan_4 — Tue, 08 Jun 2010 06:45:04 GMT

The rx_fw_discards counter means that firmware is dropping rx packets because the driver is not fast enough to retrieve these rx packets and post new buffers. The default rx ring size is 255 and you can increase it with ethtool -G to a bigger size (1020 for example). This usually will help. If flow control is not already enabled, enabling it in the NIC and switch will also help. Check with ethtool -a eth3.