How to get rid of FIN_WAIT_1

[Hoefnix]

I have a problem restarting nsrexecd (legato) on a server, its complaining in the log:
02/24/04 11:24:46 nsrexecd: Cannot bind socket to service port in configured port range on system x.
02/24/04 11:24:46 nsrexecd: unable to allocate transport; aborting

netstat -an | grep 7937 (port of nsrexecd) shows:

x.x.x.x.7937 y.y.y.y.600 32768 0 9660 0 FIN_WAIT_1

It seems that system y.y.y.y did not close its session with an ACK.

Question is, how can I force the FIN_WAIT_1 to go into TIME_WAIT without rebooting the system(it a production system)?

Thanks in advance,

Peter Geluk

Ps. When the netstat -an was done NO nsr proc's were running!

[Mark Grant]

This thread might help but Massimos answer looks scary to me!

http://forums1.itrc.hp.com/service/forums/questionanswer.do?threadId=233507

[Hoefnix]

Mark,
It looks scary to me to. According to the points in the other post it's not used.
Did somebody use it to remove a connection using:
# ndd -set /dev/tcp tcp_discon 0x
Or other suggestions are welkom.

Regards,
Peter

[Christian Maxeiner]

Find out the number 0x????? for the WAITING socket (first column in the table):
ndd -get /dev/tcp tcp_status |grep -e state -e TCP_FIN_WAIT

After that you can disconnect it like this:
ndd -set /dev/tcp tcp_discon 0x????????

[Hoefnix]

Sssst, don't tell, but the problem is on a SUN(5.6) system. So ndd does not know all the features.
We will schedule a reboot for this system or someone can tell me another solution?

Thanks for the anwsers.
Peter

[rick jones]

Peter -

While it is an anathema to me to help someone with a problem on a competing OS, here are some things to consider:

1) Upgrade your Solaris system to an HP-UX 11i v2 system.

2) Your legato software is broken. That it cannot restart and bind() to its well known port while there are old connectoins in FIN_WAIT_1 suggests that it is not calling setsockopt(SO_REUSEADDR) and thus broken.

3) FIN_WAIT_1 is an active retransmission state. A FIN has been sent, but no ACK of that FIN has been recieved. As such, any connection in that state should terminate within tcp_ip_abort_interval, or its evil-twin Solaris equivalent.

3b) Your Solaris TCP/IP stack is broken and not groking an ACK of the FIN it has sent, or is not retransmitting a FIN at all when it should.

4) There may be some ndd kludges to forcibly terminate a TCP connection, but frankly, I wouldn't suggest their use even on the OS of my competition.

[Andrew Cowan]

On some OS's kiling the inetd, and then waiting a few seconds before restarting will purge all the FIN_WAIT, CLOSE-WAIT, sockets. Its one of those things that I have found that vary from OS, and version to version.

[Hoefnix]

Rick,

Thanks for the suggestions I like the first one, but it's still a solution that causes downtime ;)

Andrew,

The inetd restart would not work because this socket was not started by the inetd. Still I did test it (could not do any harm) but it did not free up the socket.

We got a go for the reboot(msSUN) and now the legato software started correct.

Thanks all,

Peter

[rick jones]

Upgrading your Solaris 5.6 system to an 11i v2 system need not cause downtime - just requires having the systems up in parallel, perhaps prepping by adjusting the DNS TTLs way down, and then do the swap in DNS. OK, that may be a triffle simplistic, but considering you've been given the go ahead for downtime in the first place to do the reboot that you will have to do again the next time you want to restart legato... :)