Socket blocked

[Matteo_9]

We have an hp9000 with hpux 11.1
When one application die the operating system doesn't free the socket 6523, how can we force it?

[Ivajlo Yanakiev]

did you use lsof ?
It will give you open sockets and process that is connecto to them.

[Matteo_9]

I don't have lsof installed. Where i can find it?

[Jannik]

http://ftp.cerias.purdue.edu/pub/tools/unix/sysutils/lsof/binaries/hpux/

[Matteo_9]

I've installed it but it doesn't give me helpful informations:
with the command:

netstat -av |grep 6523

Hp answer me:

tcp 0 0 LARC9000.6523 .1045 FIN_WAIT_2
tcp 0 0 LARC9000.6523 .1044 FIN_WAIT_2

How can i kill the process that occupies the socket 6523?

[KapilRaj]

I hv the same prob with an aix box, i had to reboot it to get it cleared !!.

Kaps

[Jannik]

I don't know if this is related to defunct proc (then you have a code problem):

A zombie process is a process which has died and whose parent process is still running and has not wait()ed for it. In other words, if a process becomes a zombie, it means that the parent process has not called wait() or waitpid() to obtain the child process's termination status. Once the parent retrieves a child's termination status, that child process no longer appears in the process table.

You cannot kill a zombie process, because it's already dead. It is taking up space in the process table, and that's about it.

If any process terminates before its children do, init inherits those children. When they die, init calls one of the wait() functions to retrieve the child's termination status, and the child disappears from the process table.

A zombie process is not, in and of itself, harmful, unless there are many of them taking up space in the process table. But it's generally bad programming practice to leave zombies lying around, in the same way
that it's generally a Bad Thing to never bother to free memory you've malloc()ed.

[Stanimir]

You need to find the open ports both local and remote, for example:

#netstat -av | grep 1026

tcp 0 0 192.1.2.3.1026 192.4.5.6.2049 FIN_WAIT_2
.....

This means:
Local IP: 192.1.2.3 (0xc0010203)
Local Port: 1026 (0x0410)
Remote IP : 192.4.5.6 (0xc0040506)
Remote Port: 2049 (0x0801)

Using this information /hex-values are in brackets/ you can send to tcp_discon_by_addr the follow:

# ndd -set /dev/tcp tcp_discon_by_addr "c00102030410c00405060801"

Regards.

[Ivajlo Yanakiev]

This is software problem.
Ask for software support. There are bug in your application.

[rick jones]

1) FIN_WAIT_2 can exist without any process associated with the endpoint. At that stage, there really isn't a socket, but there is still a TCP endpoint.

2) A "properly" written server application will use setsockopt(SO_REUSEADDR) before attempting to bind() to its port number(s) and this will allow it to be restarted while there are still endpoints with that port number in any state "above" but not including "LISTEN"

3) The tcp_discon kludge is something that one really, Really, REALLY should not use unless you are truly, Truly, TRULY desparate.

Summary - the application is broken and requires a fix which should be applied ASAP.

Additional discussion:

FIN_WAIT_2 can also happen without the local application calling close(), but only calling shutdown(). It is now waiting for the remote TCP to send a FINished segment (TCP allows half-open connections). If the remote application is written incorrectly, it may cause TCP to send a RST (reset) segment instead. RST's are not retransmitted, so if it is lost, the side in FIN_WAIT_2 may stay there "forever."

HP-UX, when the application calls close() will initiate keepalive probes after tcp_keepalive_detached_interval units of time. If these do not elicit a response from the remote within tcp_ip_abort_interval the connection will be terminated. The application must have called close(). If the remote TCP responds with ACKs instead of say a retransmitted FIN or a re-elicited RST, the connection will remain. Some firewalls may be broken in this regard.

If an application just calls shutdown(), it really should have some sort of timer going to decide when to call close() if it receives no connection close notification from the remote. At the _very_ least it should setsockopt(SO_KEEPALIVE).