topic Re: closed fds seen as LISTENING in Operating System - HP-UX

closed fds seen as LISTENING

Samir_10 — Fri, 28 Nov 2003 02:02:58 GMT

HP-UX 11.0 9000/800

We have an interface server which listens on multiple ports. It is required to close all client sockets and base sockets at some point of time. Using netstat we found that the base sockets(listening fds)which are closed are still seen as LISTENING. Glance is however not showing these fds. Any attempt to connect to these ports are successful. Is this a bug? Is there any patch we have to apply?

Re: closed fds seen as LISTENING

Laurent Menase — Fri, 28 Nov 2003 05:34:18 GMT

It doesn't looks like to any know problem.
Do you use the socket caching?
Did you check with lsof that no process could still have this port opened? ( for instance a child process of the one which closed)
Usually that sort of things arrive when a child process still have a file handler on the socket.
For instance if you do
fd=socket()
fd1=accept(fd...)
if (fork())
{ /* child do not close the fd */
...
} else close(fd1);
In that case the socket will stay in listen state as long as a child process which had not closed it explicitely is still running

Re: closed fds seen as LISTENING

Elmar P. Kolkman — Fri, 28 Nov 2003 05:40:39 GMT

Could it be that inetd is listening on those ports ?
Try to use something like lsof to find out which process is still listening on those ports at that time...

Re: closed fds seen as LISTENING

Samir_10 — Fri, 28 Nov 2003 07:18:33 GMT

The server is not forking child processes.
lsof and glance do not show these fds in their output but netstat shows them as listening.
An attempt to 'telnet localhost portNum' is successful and netstat shows ESTABLISHED connection on these ports.

Any pointers as to what's going wrong ?

Re: closed fds seen as LISTENING

Ralph Grothe — Fri, 28 Nov 2003 07:28:00 GMT

Do the sockets belong to stand-alone servers, or are they created by servers who themselves are forked from others (e.g. like inetd).
If they are started through inetd you could increase verbosity by toggling it back and forth to/from extended connection logging mode whenever you issue
/usr/sbin/inetd -b
Then do a tail -f on your syslog.log or wherever your syslogd is writing output for facility daemon.
It will also print the PIDs of your freshly spawned servers.

Re: closed fds seen as LISTENING

Samir_10 — Fri, 28 Nov 2003 07:41:14 GMT

Ralph,

The sockets are created by a stand-laone server.

--Samir

Re: closed fds seen as LISTENING

Ralph Grothe — Fri, 28 Nov 2003 08:08:38 GMT

Then it sounds to me as if your server though being signaled to terminate is still caught in its accept() loop since you wrote that it would still establish new connect requests.
Are you sure you have sent the right process the right signal?

Re: closed fds seen as LISTENING

rick jones — Mon, 01 Dec 2003 21:07:04 GMT

While the application does not fork children for new connections and such, does the application daemonize, or fork during its startup processing?

What do you see if you take a tusc trace of the application - either during startup (be sure to set the follow forks option just to be safe :) or during operation?

As for patches to apply to an 11.0 box, well, since 11.0 first shipped in 1997, there have been some TCP/IP patches. You might want to make sure you are up on some that are recent.

Re: closed fds seen as LISTENING

Samir_10 — Mon, 01 Dec 2003 23:57:48 GMT

Rick,

No the application does not daemonize or fork
during startup.

It is observerd that on the first close of the basesocket, the netstat shows the ports as still listening.

Upon introduction of another close of the basesocket just after the first one, perror returns 'Bad file number' and netstat does not show up the ports as listening !!.

I feel we are missing on some network related patches on HP-UX 11.0.

Pl. help.

--Samir

Re: closed fds seen as LISTENING

rick jones — Tue, 02 Dec 2003 12:38:28 GMT

I have no idea if it will fix your problem, and do be _sure_ to read all the warnings and such, but here is a list of the latest ARPA transport patch and its dependencies for 11.0.

PHCO_23651 fsck_vxfs(1M) cumulative patch
PHCO_24437 LVM commands cumulative patch
PHCO_27375 cumulative SAM/ObAM patch
PHCO_29380 user/group(add/mod/del)(1M) cumulative patch
PHKL_18543 PM/VM/UFS/async/scsi/io/DMAPI/JFS/perf patch
PHKL_20016 2nd CPU not recognized in G70/H70/I70
PHKL_23409 NFS, Large Data Space, kernel memory leak
PHKL_28150 LVM Cumulative Patch w/Performance Upgrades
PHKL_28593 VxFS 3.1 cumulative patch: CR_EIEM
PHKL_29385 IDS/9000; syscalls; eventports; dup2() race
PHKL_29434 POSIX AIO;getdirentries;MVFS;rcp;mmap/IDS;
PHKL_29648 Probe,IDDS,PM,VM,PA-8700,AIO,T600,FS,PDC,CLK
PHNE_27902 Cumulative STREAMS Patch
PHNE_28538 cumulative ARPA Transport patch