How we can troble shoot network card in Linux using the details of "netstat -s"

[senthil_kumar_1]

Hi All,

How we can troble shoot network card in Linux using the details of "netstat -s"

Example:

# netstat -s
Ip:
80615385 total packets received
0 forwarded
0 incoming packets discarded
80209314 incoming packets delivered
3574069352 requests sent out
1 fragments dropped after timeout
457121 reassemblies required
150187 packets reassembled ok
1 packet reassembles failed
Icmp:
965396 ICMP messages received
35 input ICMP message failed.
ICMP input histogram:
destination unreachable: 4340
source quenches: 1412
echo requests: 959642
echo replies: 2
1001434 ICMP messages sent
0 ICMP messages failed
ICMP output histogram:
destination unreachable: 41792
echo replies: 959642
Tcp:
6142131 active connections openings
12631903 passive connection openings
17 failed connection attempts
217181 connection resets received
269 connections established
3951608860 segments received
3294965247 segments send out
4529693 segments retransmited
4 bad segments received.
5553226 resets sent
Udp:
413966376 packets received
41400 packets to unknown port received.
7908146 packet receive errors
278104300 packets sent
TcpExt:
1694491 invalid SYN cookies received
4375774 resets received for embryonic SYN_RECV sockets
343 packets pruned from receive queue because of socket buffer overrun
ArpFilter: 0
2905140 TCP sockets finished time wait in fast timer
1572 time wait sockets recycled by time stamp
9421058 delayed acks sent
30962 delayed acks further delayed because of locked socket
Quick ack mode was activated 72345 times
36234902 packets directly queued to recvmsg prequeue.
937443666 packets directly received from backlog
2941751753 packets directly received from prequeue
395884522 packets header predicted
31009938 packets header predicted and directly queued to user
TCPPureAcks: 671134925
TCPHPAcks: -1815547433
TCPRenoRecovery: 527842
TCPSackRecovery: 32679
TCPSACKReneging: 0
TCPFACKReorder: 46
TCPSACKReorder: 30
TCPRenoReorder: 15633
TCPTSReorder: 90
TCPFullUndo: 222
TCPPartialUndo: 1981
TCPDSACKUndo: 0
TCPLossUndo: 1259
TCPLoss: 4583
TCPLostRetransmit: 5
TCPRenoFailures: 3341
TCPSackFailures: 1154
TCPLossFailures: 47
TCPFastRetrans: 3346455
TCPForwardRetrans: 4118
TCPSlowStartRetrans: 342528
TCPTimeouts: 795297
TCPRenoRecoveryFail: 22187
TCPSackRecoveryFail: 437
TCPSchedulerFailed: 1117
TCPRcvCollapsed: 23982
TCPDSACKOldSent: 71643
TCPDSACKOfoSent: 2
TCPDSACKRecv: 206
TCPDSACKOfoRecv: 0
TCPAbortOnSyn: 0
TCPAbortOnData: 1457953
TCPAbortOnClose: 91
TCPAbortOnMemory: 0
TCPAbortOnTimeout: 787
TCPAbortOnLinger: 0
TCPAbortFailed: 0
TCPMemoryPressures: 0


My Questions:

1)What are fields needs to be checked?

2)What is the optimum value for all important fields?

3)How can know that there is issue with network back plane?

4)How we can check that there is issue with network card (ethernet card)?

5)When we have to check this statistics?

[Ishwar_1]

netstat -s :- Display summary statistics for each protocol.

1)What are fields needs to be checked?
Solution:-
Network Bottleneck you need to illustrate whats the problem related to.
=> Ensure that the network card configuration matches router and switch configurations (for example, frame size).
=> Modify how your subnets are organized.
=> Use faster network cards.
=> Tune the appropriate IPV4 TCP kernel parameters. Some security-related parameters can also improve performance.
=> If possible, change network cards and recheck performance.
=> Add network cards and bind them together to form an adapter team, if possible.


2)What is the optimum value for all important fields?
It depends upon the Requirement an type of application you are using. Incase for increasing the bandwidth you can go channel bonding of the Network Card. Increase the Network Buffer, packet queues. Decrease interrupts


3)How can know that there is issue with network back plane?
check the logs, dmesg and you can replace it and check

4)How we can check that there is issue with network card (ethernet card)?
ifconfig output Show you the status whether the network card is up or not
Or
# /etc/init.d/network status

5)When we have to check this statistics?
Try to automate most of the things by writing script and Customize your monitoring tool to alert whenever it reaches threshold

[Steven E. Protter]

Shalom,

Your stats do not look all that bad. I packet dropped, though the figure should be zero.

1)What are fields needs to be checked?
Fields reporting errors or dropped packets.


2)What is the optimum value for all important fields?
Zero for errors, drops and malformed packets, large numbers for good stuff.

3)How can know that there is issue with network back plane?

Lots of errors. Way more than I see.

4)How we can check that there is issue with network card (ethernet card)?

You could boot the server into diagnostic mode and test the NIC

5)When we have to check this statistics?

I would look at these every week or so, or better yet, write a script that checks for errors regularly and emails you.

SEP

[senthil_kumar_1]

Hi Steven E. Protter,

Could you please share your scripts that checks statistics of network card and sends mail if errors happen?

Could you please share any doc about optimum values for all important fields?

[Steven E. Protter · Accepted solution]

Shalom,

Didn't know I had scripts that looked at netstat, but I should.

See:
http://hpux.ws/system.perf.sh
# Requires a port to Linux....


Here is a really fast try at a script using
filename=/tmp/$$.net.dat
netstat -s > $filename
tcpimeout=$(grep TCPAbortOnTimeout $filename |awk '{print $2}')


# Set other values you think are relevant here.

Then write a report of these values to a file or email and send it only when there are errors.

You will also want to reset the statistics with netstat (see man page). So you don't get the same error.

I'd build the script and run it daily.

Closer look shows some stuff that concerns me.


TCPAbortOnTimeout: 787
937443666 packets directly received from backlog

I just checked a CentOS vm I built yesterday and see no errors, but this system does no really significant work.

SEP