Network tuning (challenge, at least for me)

[Manuel Urena]

Hi all,

Let me explain you my scenario.

Two unix servers (one sunfire v120, the other an hp N4000-55 box).

Both interfaces configured as fast ethernet.

This is the lanadmin output in the hp box:

LAN INTERFACE STATUS DISPLAY
Wed, Oct 4,2006 10:45:03

PPA Number = 0
Description = lan0 HP PCI 10/100Base-TX Core [100BASE-TX,HD,AUTO,TT=1500]
Type (value) = ethernet-csmacd(6)
MTU Size = 1500
Speed = 100000000
Station Address = 0x306e03d801
Administration Status (value) = up(1)
Operation Status (value) = up(1)
Last Change = 18581897
Inbound Octets = 555962
Inbound Unicast Packets = 9245
Inbound Non-Unicast Packets = 0
Inbound Discards = 0
Inbound Errors = 0
Inbound Unknown Protocols = 0
Outbound Octets = 18698982
Outbound Unicast Packets = 21749
Outbound Non-Unicast Packets = 1
Outbound Discards = 0
Outbound Errors = 0
Outbound Queue Length = 0
Specific = 655367

Ethernet-like Statistics Group

Index = 1
Alignment Errors = 0
FCS Errors = 0
Single Collision Frames = 2
Multiple Collision Frames = 0
Deferred Transmissions = 2
Late Collisions = 1670
Excessive Collisions = 0
Internal MAC Transmit Errors = 0
Carrier Sense Errors = 0
Frames Too Long = 0
Internal MAC Receive Errors = 0

Below the netstat -ni of the HP BOX:

root# netstat -ni
Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll
lan0 1500 172.16.1.0 172.16.1.49 8499922 0 3996083 0 2
lo0 4136 127.0.0.0 127.0.0.1 25321 0 25321 0 0


SUN BOX:
root# netstat -ni
Name Mtu Net/Dest Address Ipkts Ierrs Opkts Oerrs Collis Queue
lo0 8232 127.0.0.0 127.0.0.1 73 0 73 0 0 0
eri0 1500 10.10.71.0 10.10.71.74 187927 0 1630 1 0 0
eri1 1500 172.16.1.0 172.16.1.31 40841 3253 22382 5 0 0

root# ndd /dev/eri instance
1

root# ndd /dev/eri link_status
1

root# ndd /dev/eri link_mode
1

root# ndd /dev/eri link_speed
1


I am utilizing ttcp (I've attached the source of this classic tool)

SUN BOX:
root# ~/UXTools/ttcp.sun4 -r -s -l1500
ttcp-r: nbuf=1024, buflen=1500, port=2000
ttcp-r: socket
ttcp-r: accept
ttcp-r: 0.0user 0.1sys 10:38real 0% 0i+0d 0maxrss 0+0pf 6755+24csw
ttcp-r: 15000000 bytes processed
ttcp-r: 0.12 CPU sec = 122070 KB/cpu sec, 976562 Kbits/cpu sec
ttcp-r: 638.645 real sec = 22.9367 KB/real sec, 183.494 Kbits/sec


HP BOX:

root# ./ttcp -t -s -l1500 -n10000 sundev
ttcp-t: nbuf=10000, buflen=1500, port=2000
ttcp-t: socket
ttcp-t: connect
ttcp-t: 0.0user 0.1sys 10:37real 0% 0i+0d 0maxrss 0+0pf 555+2csw
ttcp-t: 15000000 bytes processed
ttcp-t: 0.16 CPU sec = 91552.7 KB/cpu sec, 732422 Kbits/cpu sec
ttcp-t: 637.405 real sec = 22.9814 KB/real sec, 183.851 Kbits/sec


As you seen the throughput is extremely slow for servers talking via a crossover cable under fast ethernet!

If anybody could find what could be wrong, please tell me. I am almost frustrated with this.

Is there any parameters I should tweak with?

Thanks a lot in advance,

Manuel

[spex]

Hi Manuel,

> [100BASE-TX,HD,AUTO,TT=1500]

You might want to start by setting lan0 to full duplex, and making sure eri1 on the v120 is set to the same.

PCS

[rick jones · Accepted solution]

That the lanadmin shows the NIC was autoneg, half-duplex, and experiencing _late_ collisions suggests that the other end of the link was improperly (IMO) hardcoded. set the other end of the link to autoneg and things should be much better - modulo the wimpy nature of a V210 :)

$ cat usenet_replies/duplex
How 100Base-T Autoneg is supposed to work:

When both sides of the link are set to autoneg, they will "negotiate"
the duplex setting and select full-duplex if both sides can do
full-duplex.

If one side is hardcoded and not using autoneg, the autoneg process
will "fail" and the side trying to autoneg is required by spec to use
half-duplex mode.

If one side is using half-duplex, and the other is using full-duplex,
sorrow and woe is the usual result.

So, the following table shows what will happen given various settings
on each side:

Auto Half Full

Auto Happiness Lucky Sorrow

Half Lucky Happiness Sorrow

Full Sorrow Sorrow Happiness

Happiness means that there is a good shot of everything going well.
Lucky means that things will likely go well, but not because you did
anything correctly :) Sorrow means that there _will_ be a duplex
mis-match.

When there is a duplex mismatch, on the side running half-duplex you
will see various errors and probably a number of _LATE_ collisions
("normal" collisions don't count here). On the side running
full-duplex you will see things like FCS errors. Note that those
errors are not necessarily conclusive, they are simply indicators.

Further, it is important to keep in mind that a "clean" ping (or the
like - eg "linkloop" or default netperf TCP_RR) test result is
inconclusive here - a duplex mismatch causes lost traffic _only_ when
both sides of the link try to speak at the same time. A typical ping
test, being synchronous, one at a time request/response, never tries
to have both sides talking at the same time.

Finally, when/if you migrate to 1000Base-T, everything has to be set
to auto-neg anyway.

[Manuel Urena]

Thank you so much PCS and Rick.

Rick thank you so much for your insightful and detail explanation.

I still have a question I may know how to tweak the NIC parameters on the SUN box. (Utilizing ndd /dev/eri) But I don't know how to do this on HP-UX.

Could you guys please tell me how to change the parameter (HD/FD, AUTONEG, etc...) with lan0?

Thanks a lot.

Manuel

[Manuel Urena]

Hi again,

In my previous posting ( Could you guys please tell me how to change the parameters (HD/FD, AUTONEG, etc...) with lan0? )

I forgot to specify that I would like to know both things
-How to make these changes dynamically (i.e. w/o rebooting machine) and also save my changes making them persistent after a reboot.

Thanks a lot,

Manuel

[IT_2007]

you can change setting dynamically but you it says network problems for 11seconds..

lanadmin -x 0

which will show details for lan0

lanadmin -X 100FD 0

which will set lan0 with 100 Full Duplex auto off.

you can set these permanent by adding proper values in files /etc/rc.config.d

hpbtlanconf
hpetherlanconf
hpigelanconf

Use vi editor to edit the above files. But you need to edit only one whichever LAN card you are using it.

[rick jones]

Since the HP-UX box is already set to autoneg, there is nothing for you to change there. What you need to change is the other end of the cable - either the v210 if they are back to back, or your switch port - to be autoneg.

[Manuel Urena]

Rick,

I just did as you said, and now everything looks normal.

By the way this is what I did on the sun box:

ndd -set /dev/eri instance 1
ndd -set /dev/eri adv_autoneg_cap 1

I asked the question of how to configure lan0 parameters/settings (both dynamically and saving the changes permanently) because I suspect I will have to tweak with other lan interfaces on other hp-ux machines.

Thanks a lot Rick. And thanks much also to everybody who took their time to help me with this issue.

Manuel

[Kevin Wright]

I would suggest you hardcode each card to FD.