topic network slow in Operating System - HP-UX

network slow

kholikt — Fri, 31 May 2002 02:39:33 GMT

I have two hpux machine that connect to a cisco catalyst 2950 switch. We have set the network card to Full duplex, the port of the switch was also configured to Full duplex. But the network from these two machine are very slow. Even ftp a 7MB file across this two machine will take a long time. Because of this issue, when I put the omniback agent to the other machine it took about 2 hours to complete. Is there any particular reason? Where can I find all network driver patch?

Re: network slow

Steven Sim Kok Leong — Fri, 31 May 2002 02:59:14 GMT

Hi,

Check your switch port statistics and your HP-UX server NIC statistics:

# netstat -s

Run lanadmin as well to check your LAN card health.

# lanadmin

Perhaps you can show us the output for both commands with respect to the NIC interface in question.

If it only occurs with either upstream or downstream (but not both), it is likely that your NIC card or switch port has not been hardcoded to full-duplex. Remember to disable auto-negotiation.

Hope this helps. Regards.

Steven Sim Kok Leong

Re: network slow

kholikt — Fri, 31 May 2002 03:24:09 GMT

one more thing I want to add.

I try to ftp a file from the unix machine to one of the NT machine that connect to the same switch is very fast. It only took less than 10 seconds. From hpux to hpux this is very slow. It took more than 10 minutes to get the file transfer

Re: network slow

kholikt — Fri, 31 May 2002 03:26:45 GMT

Re: network slow

T G Manikandan — Fri, 31 May 2002 03:44:33 GMT

Hello,
I assume there is a mismatch of speed between the hp machine and the hub/switch.Also check the other hpux machine NIC speed.

Thanks

Re: network slow

Gnananandhan — Fri, 31 May 2002 04:48:37 GMT

Hi,
Pls check the speed of both the server and client m/c speed. One more thing is if it is across switches, check the settings of each switch. And also check for any port in each switch is making an heavy traffic.

Regards,
Gnana A.

Re: network slow

kholikt — Fri, 31 May 2002 05:45:57 GMT

Hi,

From the netstat -p tcp output I have found something very interesting.

tcp:
93198244 packets sent
85116674 data packets (3890717183 bytes)
42830 data packets (50051924 bytes) retransmitted
8084826 ack-only packets (1688373 delayed)
2490 URG only packets
35 window probe packets
5514 window update packets
586634 control packets
92380043 packets received
77634253 acks (for 3890740464 bytes)
147951 duplicate acks
0 acks for unsent data
83090236 packets (131548687 bytes) received in-sequence
21 completely duplicate packets (26406 bytes)
8783 packets with some dup, data (2812157 bytes duped)
50433 out of order packets (26925711 bytes)
0 packets (0 bytes) of data after window
1158 window probes
14360122 window update packets
745 packets received after close
40 segments discarded for bad checksum
0 bad TCP segments dropped due to state change
172497 connection requests
95554 connection accepts
268051 connections established (including accepts)
368418 connections closed (including 100530 drops)
99316 embryonic connections dropped
77385442 segments updated rtt (of 77385442 attempts)
27690 retransmit timeouts
948 connections dropped by rexmit timeout
35 persist timeouts
50001 keepalive timeouts
50001 keepalive probes sent
0 connections dropped by keepalive
2 connect requests dropped due to full queue
98723 connect requests dropped due to no listener

The retransmit timeouts seems to be very high. I was thinking is this possible due to default socket buffer size for a hpux is 32k instead of 16k. If this is the case how could we use the ndd command to reduce the size

Re: network slow

Steven Sim Kok Leong — Fri, 31 May 2002 08:56:15 GMT

Hi,

If you want to reduce the window size from 32k to 16k, then:

# ndd -get /dev/tcp tcp_xmit_hiwater_def
32768
# ndd -set /dev/tcp tcp_xmit_hiwater_def 16384

At first glance, your netstat outputs do not look too healthy, with a quite a number of drops etc. Inbound errors are relatively small compared with the total inbound bytes received.

As the statistics are cumulative, I would suggest that you monitor netstat again at 3 hours interval and identify the error counters (and rate) that are consistently increasing.

To verify the throughput, you can use iperf (I have found it very accurate) to measure the max. throughput both downstream and upstream:

http://dast.nlanr.net/Projects/Iperf/

Hope this helps. Regards.

Steven Sim Kok Leong

Re: network slow

Sebastian Galeski_1 — Fri, 31 May 2002 09:50:04 GMT

Hello
My advise is: go to www.netperf.org, download netperf software and run test. One of the most important test is to do:
netperf -H loopback_address i.e.
netperf -H 127.0.0.1

It will let You know if localy network seems to work properly, sometimes network work poor because thereis CPU or memory issue.

hope it help Seba

BTW ftp is not good test to say if network work well or not

Re: network slow

sven verhaegen — Fri, 31 May 2002 10:48:21 GMT

well

a fairly simple test could reveile more info , just ping tot he router first and note if it looses packets or if the general delay time is high , if so then the problem is local machine or router related , if that is ok then try to ping the other side and see if there is any packet loss or delay in reply , I've seen a lot of performance issue up till now most of them as tated in most posts here related to duplex setting wrong bewteen switch/srouter nd sender or received but hat is easy to prove , if it is duplex related a revers direction data transfer shouldn't suffer from the problem at all e.g ftp put is slow but ftp get is fast

there are some more marginal stuff that could be involved here , one t hing I know about 11.x version is that it runs a check on the input buffers now , called icmp source quenceing .. it has been noted that in some marginal cases this check does cause performance issue , but then you should be seeing these source quence messages either in the syslog or on the router. There is a way to turn this of (it will not alter any functionality as hp-ux 10.20 didn't even have the feature and worked fine without , thsi an optional feature)
how to disable it :

#ndd -set /dev/ip ip_send_source_quench 0

doing this might just fix is swiftly , if so you need to make this permanent as at reboot the setting wil disappear so changes are required in the /etc/rc.config.d/nddconf file

modify the /etc/rc.config.d/nddconf file as follows:

TRANSPORT_NAME[X]=ip
NDD_NAME[X]=ip_send_source_quench
NDD_VALUE[X]=0

Where X is the next logical numerical sequence in a table
of values, with X starting at 0.

Re: network slow

sven verhaegen — Fri, 31 May 2002 10:51:36 GMT

I forgot to add that you should look at the latest ARPA transport pach for the OS version you are running ,some older version where recalled for a performance issuesso putting the latest should be ok , also put the latest LAN cumulative patch and the latest patch specific for your lancard in (if you didn't already), I don't have an idea what type lancard it is but al the patches are easily downloaded from the ITRC patches site , make sure to check for dependency patching

Re: network slow

Grahame Wright — Fri, 26 Mar 2004 09:10:35 GMT

kholikt, I'd like to know if you ever found a solution to your problem. We have 4 new rp5470s plugged into a cisco 2950 and we're having the exact same network slowness issues. We also have 2 K570s plugged into the same switch and those communicate fine with each other. It's definitely not a Duplex issue since all of the ports on the switch are manually set to 100 Full, and so are the NICs on the HPUX machines. Any help you can give would be most appreciated!

Thanks