high number of retransmitting -- netstat -p tcp

[Kristopher March]

I am experiencing a high number of retransmits under our netstat output. No one has really paid any attention to these numbers until after we were moved from one failing network switch(FDDI) to another, newer switch(ethernet). I've been trying to track down the cause of these retransmits with no luck in indentifying what's causing them. I understand, for the most part, why they happen.

The attached doc has more information. One item I was tracking down last week was if the decrease in our MTU has any effect on this? It was once up in the 4000 range, but now down to 1500.

The server is HP-UX 11, running an old 100MB Combo interface card in a V class. I'm not sure if I need to tune the current interface for these new settings. Any suggestions? It's also hard to tell if I'm truly running at 100MB FD.

LAN INTERFACE STATUS DISPLAY
Mon, Dec 20,2004 12:51:31

PPA Number = 1
Description = lan1 Hewlett-Packard 10/100 TX Full-Duplex TT = 1500
Type (value) = ethernet-csmacd(6)
MTU Size = 1500
Speed = 100000000
Station Address = 0x60b07ac2ab
Administration Status (value) = up(1)
Operation Status (value) = up(1)
Last Change = 167725
Inbound Octets = 1998166485
Inbound Unicast Packets = 10966804
Inbound Non-Unicast Packets = 186183
Inbound Discards = 0
Inbound Errors = 0
Inbound Unknown Protocols = 0
Outbound Octets = 4231290580
Outbound Unicast Packets = 12254061
Outbound Non-Unicast Packets = 223076
Outbound Discards = 0
Outbound Errors = 0
Outbound Queue Length = 0
Specific = 655367

SAM gives me an error about the drivers bound to this card.

" The driver specific configuration file in /etc/rc.config.d doesn't support the configuration of Speed, Duplex, and Autonegotiation parameters through SAM. Please copy the latest configuration file from /user/newconfig/etc/rc.config.d directory to enable his or contact your support personnel for further assistance."

Is there another way to verifty these settings? Or does the Description portion of my lanadmin command make it obvious?

Thanks.

[Paul Eadington_1]

Going from the output above I'd say you were running at 100Mb from the speed output.

To verify the Duplex setting try the following.

lanadmin -x 1 (where 1 is the NMID of the interface your checking)

As long as the driver/hardware supports this you should get something like the following.

root@host# lanadmin -x 1
Current Speed = 100 Full-Duplex Auto-Negotiation-OFF

[Kristopher March]

Thanks.

lanadmin shows me the settings:

# lanadmin -s 1
Speed = 100000000
# lanadmin -x 1
Current Speed = 100 Full-Duplex Auto-Negotiation-ON


What's the difference between ON and OFF with the Auto Neg?

[Ron Kinner]

I think if it tells you in the description that it is FULL that that is what it is running.

If you had a duplex mismatch problem with this you would see a lot of errors on this end and collisions on the other end.

The Ethernet connection looks pretty clean.

The MTU is just the difference between Ethernet and FDDI. The major effect is that you do have more packets to handle if you are sending a lot of data. By itself this is not a problem.

It appears that somewhere in your network you are dropping packets and it's not on the 100 Full link. What does the network look like? Do you have a WAN link somewhere? Any bottlenecks where the connection is less than 100 Full or perhaps an interswitch trunk that is serving multiple 100 FUll circuits with a single 100 Full connection?

Ron

[Paul Eadington_1]

Your card is set to Auto negotiate with whatever it is connected to at the other end of the cable. In this case whatever it is is set to 100FD.

In my example we have the card and the switch both hard set to be 100 Full Duplex, only because we have seen issues in the past where connections negotiate themselves to HD or 10Mb.

[rick jones]

Alas, you needed to include the last bit of lanadmin output :) If you are in FD mode, then FCS or CRC errors may indicate a duplex mismatch. If you are in HD mode, then _LATE_ collisions may indicate the same. Otherwise, your duplex is likely just fine and dandy, and my personal preference is to NOT hardcode unless and until it is shown that neither the NIC nor the switch can be made to work correctly.

I'm guessing you do not have a duplex mismatch because I see no inbound or outbound errors in your lanadmin stats. That being the case, the packet losses are elsewhere. You will need to start checking the stats on the switch(es) router(s) and other system(s).

Also, the absolute number of retransmits isn't really the important metric - it is the relative number. And beyond that, how many of them are "slow" (aka timeout based) versus fast. ftp://ftp.cup.hp.com/dist/networking/briefs/annotated_netstat.txt .

The change in MTU should not be an issue for retransmits. It will be an issue for the efficiency of data transfer - it now takes ~3 times as many trips up and down the protocol stack to transfer a given quantity of data than it did before.

Is the rest of your network still FDDI, or has it all gone to Ethernet as well? There aren't any of those ne'er-do-well FDDI to Ethernet "bridges" in your network are there?

[Kristopher March]

No, there aren't any duplex mismatches.

Two other similar machines that were moved from the fddi switch aren't experiencing these high number of retransmits.

And I believe this is a direct connect from the server to the switch. There isn't anything in between.

The webserver that this Vclass talks to is also running a high number in the retransmit area. The web box is Sun running 5.9. I ran a snoop between the two machines in question and am looking at the data now. I also have our firewall team looking at this same data and they don't see anything out of the ordinary.

Thanks for the doc. I've been looking at that for over a week now - learning new stuff.

[doug mielke]

Early in the full duplex days there was an incompatability between the group of vendors that developed full duplex (4 of them, including Compaq?) and those that came later, like 3Com.

Since you're using an old card, this could be what you're facing.

The answer for us was to always turn autoneg. off on both sides, switch and server, and set speeds / duplex manually.