TCP Performace

[Eduardo_63]

I've Oracle Applications on HP-UX. The apache port is 8000.
I see much TIME_WAIT and FIN_WAIT_2


$ netstat -an | grep tcp | grep 8000 | awk '{print $6}'| sort | uniq -c

298 CLOSE_WAIT
3 ESTABLISHED
614 FIN_WAIT_2
2 LISTEN
572 TIME_WAIT

It is a behavior normal ?
It behavior has relation with KeepAlive=OFF parameter of Apache ?


Other :

$ netstat -an | grep tcp | grep 8000 | awk '{print $6}'| sort | uniq -c
298 CLOSE_WAIT
6 ESTABLISHED
616 FIN_WAIT_2
2 LISTEN
1004 TIME_WAIT

any suggest, please ?

Thanks,

[Mel Burslan]

Eduardo,

I am not a big fan of messing with ndd settings and unless you are experiencing severe performance problems I discorage you from messing with them either. Having said that, if you are having problems and want to know if they are ndd related, you may post the output from running this little command sequence to tell nice people of itrc who understnad ndd much better than I do, what your setting are so that they can tell you something is wildly out of order. Here is the command sequence:

for i in `ndd -get /dev/tcp ?| grep write | awk {'print $1'}`
do
echo $i" ----> " `ndd -get /dev/tcp $i`
done

just copy and paste it into your terminal window and it should generate an output about 2-screens long. cut and paste that output here please.

[Eduardo_63]

Thank you very much for your hellp!!!
I uploaded the output.

[Mel Burslan]

Looking at this output, i can say that you or anybody else did not make any changes to the default /etc/rc.config.d/ndd.conf file, and most probably it is nothing but comment lines all over. Which is a good place to start from.

Now, do you have a serious performance degradation issue or are you getting any errors in the syslog or dmesg output regarding sockets and such ? If so, can you provide some examples ?

[Eduardo_63]

Hi Mel,

The environment is new, it has 5 month, it is an Oracle Applications 11i. I don't have
yet measurements of performance, but I am worried for the TIME_WAIT and FIN_WAIT_2 stat. The environmet is working with half users today, at 2 months the users number will be duplicated.

In accordance with the documentation of Oracle these parameters would lack changed:
tcp_ip_abort_interval=60000
tcp_keepalive_interval=90000

I uploaded the syslog.log

Thank you very much.
Best regards,
Eduardo.

[Mel Burslan]

Well, from the syslog, I have a gut feeling that this server of yours is being used as a DHCP server of some sort to your network ? Am I right ? If not, there is a major configuration issue with this man bootp requests coming in. Your server may not be able to handle that load as well as oracle app server load if the number of users for this oracle server double in the future. keep a window with glance running on it on your desktop for this system and see how the CPU/DISK/MEMORY/SWAP utilization bars are doing on a normal day. You will see where your bottlenecks are.

to configure the ndd parameters, that is an easy one, just edit /etc/rc.config.d/nddconf and following the examples which are commented out, create two blocks of data with the two parameters you want to modify, and assign them to the values you need as follows:

TRANSPORT_NAME[0]=tcp
NDD_NAME[0]=tcp_ip_abort_interval
NDD_VALUE[0]=60000

TRANSPORT_NAME[1]=tcp
NDD_NAME[1]=tcp_keepalive_interval
NDD_VALUE[1]=90000


make sure the index number in square brackets are unique for each parameter. If 0 and 1 are used take the next available one in the sequence.

after saving this file, run this command

ndd -c

to re-read the modifications you just made.

hope this helps.

[Eduardo_63]

Mel,

The server not is DHCP, it only is HTTP server.
About workload, the CPU is 65 idle and the memory (with top):
Memory: 2173136K (1898272K) real, 3901892K (3567952K) virtual, 1092252K free Page# 1/8

Thank very much for your help.

[Todd Whitcher]

Hi Eduardo,

FIN_WAIT_2 and TIME_WAIT are valid port states. FIN_WAIT_2 means that you have sent your FIN and the other end has acknowleded it but you are waiting for the other end of the connection to recognize it has received an end-of-file notification and close its end of the connection, which will send a FIN back to you. So only when the process at the other end does the close will your end move from FIN_WAIT_2 to TIME_WAIT.

TIME_WAIT aka 2MSL (Maximum Segment Lifetime). It lets TCP resend the final ACK in case this ACK is lost (in which case the other host will time out and retransmit its final FIN).

For HPUX ports will stay in TIME_WAIT for the value of tcp_time_wait_interval which is 1 minute by default.

For ports in FIN_WAIT_2 there are also default timers to clear them up if an application is behaving badly. You should typically not have to modify those either.

The TCP stack keepalive timers will activate for connections in FIN_WAIT_2 after the tcp_keepalive_detached_interval expires(default 2 minutes). So eventually FIN_WAIT_2's will be cleaned up if the application is not performring correctly.

There are some good discussions in the itrc forums on this topic.

http://www4.itrc.hp.com/service/james/search.do?searchtext=FIN_WAIT_2&searchcriteria=allwords&searchtype=SEARCH_FORUMS&rn=25&presort=rank&x=23&y=7


If you are having performance issues its most likely not due to FIN_WAIT_2's or TIME_WAIT states.

Here is a an annotated ndd resource from Rick Jones

ftp://ftp.cup.hp.com/dist/networking/briefs/annotated_ndd.txt


Hope that helps

[Bill Hassell]

Your server is defintely running as a bootp server--that's what all those messages are in syslog. You may not have intended for this to be the case, but unfortunately, bootpd is enabled by default on a new system. Just edit /etc/inetd.conf and comment the bootpd line like this:

## bootps dgram udp wait root /usr/lbin/bootpd bootpd

Then signal inetd to re-read the config file:

inetd -c

That will eliminate the bootp messages in syslog.log.

[rick jones]

Todd's got it pretty well covered. I will just reiterate that 600+ FIN_WAIT_2's are probably _not_ a problem, nor are the TIME_WAIT's. HP-UX, or any "modern" TCP/IP stack can handle orders of magnitude more of those without difficulty.