tcp CLOSE_WAIT

[Asdrubal Pichardo]

I'm running a server application which listens to port 443 on 2 HP-UX 11.00 L2000 boxes. These machines are configured with a load balancer, in order to better distribute the load. The load balancer periodically has to check the status of the port (443) on both machines, and for this it issues a tcp connect on this port to check if the application is active. After a while, my application (running on port 443) stops to work, and it seems we are reaching some kind of limit regarding tcp connections. If I issue the command "netstat -an | grep 443" I see there are many tcp connections with state CLOSE_WAIT. I've tried changing the tcp parameters "tcp_keepalive_internval" and "tcp_ip_abort_interval". Specifically I issued:
# ndd -set /dev/tcp tcp_keepalive_interval 600000
# ndd -set /dev/tcp tcp_ip_abort_interval 2000

This does not solved the problem. Rebooting the machine solves the problem, but obviously this is not a solution! Any help would be greatly appreciated.

[Alex Glennie]

sounds as if the loadbalancer or the application is leaving child processes around ?

have you used lsof utility to investigate further ?

[Stefan Farrelly]

use the command; netstat -a | grep tcp | grep -i close to see the open tcp sockets and more importantly it will show the IP number of who/what HP-UX is waiting on to close the socket. This IP will either be that of the load balancer or some other device. To have the tcp socket finally close you either need to reboot, as youve worked out, or get the device whose IP is listed from netstat to reboot, that should also finally close the socket. Ive seen this at my current site with users PC's using sql to connect to a db on our L-class, whenever their PC dies, or their application crashes, it leaves a stuck socket, and unless their PC is rebooted the socket is left open on our HP server until we reboot it.

[Asdrubal Pichardo]

Thanks all for the replies! The problem is solved, it seems the application was doing something wrong. Now all the tcp handshakes from the load balancer are correctly closed by the application running on port 443.