Inetd Hanging

[Paul Murray_4]

I'm having a little trouble with inetd on one of my HP-UX 11.0 servers.

Even though the configuration files (/etc/inittab, /etc/inetd.conf and /etc/rc.config.d/netdaemons) and versions of the binaries match the other 11.0 servers on our network, this one particular box is showing rather strange behaviour.

When restarting inetd, inetd hangs until you issue the "inetd -b" command - it then works normally.

Restarting the process (and even rebooting the server) seems to make no difference - inetd still hangs until it gets the -b switch !!

HELP !?!?!?!?!

[Alex Lavrov.]

Do you have any entries in syslog?

[Paul Murray_4]

Hi Alex - the only syslog messages written, hit the log after the "inetd -b" command is issued (before this, no messages are logged at all).

However, the messages after issuing "inetd -b" are :-

Apr 26 08:53:08 lndvs101 syslog: inetd sendsig: entered, sig=4
Apr 26 08:53:08 lndvs101 inetd[7227]: Extended connection logging enabled
Apr 26 08:53:08 lndvs101 syslog: inetd sendsig: exiting
Apr 26 08:53:08 lndvs101 inetd[7227]: protocol = tcp
Apr 26 08:53:08 lndvs101 inetd[7227]: telnet/tcp: Added service, server /usr/lbin/telnetd
Apr 26 08:53:08 lndvs101 inetd[7227]: ftp/tcp: Added service, server /usr/lbin/ftpd
Apr 26 08:53:08 lndvs101 inetd[7227]: login/tcp: Added service, server /usr/lbin/rlogind
Apr 26 08:53:08 lndvs101 inetd[7227]: shell/tcp: Added service, server /usr/lbin/remshd
Apr 26 08:53:08 lndvs101 inetd[7227]: exec/tcp: Added service, server /usr/lbin/rexecd
Apr 26 08:53:08 lndvs101 inetd[7227]: swat/tcp: Added service, server /opt/samba/bin/swat
Apr 26 08:53:08 lndvs101 inetd[7227]: cvspserver/tcp: Added service, server /usr/local/bin/cvs-server
Apr 26 08:53:08 lndvs101 inetd[7227]: rsync/tcp: Added service, server /usr/local/bin/rsync
Apr 26 08:53:08 lndvs101 inetd[7227]: protocol = tcp
Apr 26 08:53:08 lndvs101 inetd[7227]: Configuration complete
Apr 26 08:53:08 lndvs101 inetd[7227]: About to select()...
Apr 26 08:53:10 lndvs101 syslog: inetd sendsig: entered, sig=4



Thanks,
Paul.

[Alex Lavrov.]

If you have tusc, you can try and look what system calls make inetd hang with:

tusc -f


Do you have any additional lines that you added to inetd.conf? (even if they exist on other systems, it's still can cause problems here)

[Joseph Loo]

hi,

the -b option is a toggle which turns debug on and off. what is option running on inetd"

# ps -ef|grep -i inetd

also, r there the patches for the other 11.0 similar to this server.

regards.

[Henk Geurts]

is the latest inetd patch (PHNE_28312)installed ?

regards.
H

[Alex Lavrov.]

And another question:
How do you restart inetd?

[Paul Murray_4]

The current version of inetd is :-

root@lndvs101..# what /usr/sbin/inetd
/usr/sbin/inetd:
Copyright (c) 1983 Regents of the University of California.
Revision: 1.12.214.3 Tue Aug 1 10:30:45 GMT 2000
Patch id: PHNE_21835


When restarting, I've tried using the stopping and starting using the /sbin/init.d/inetd script, and also by running "inetd -k" and "/usr/sbin/inetd -l" commands manually. Then, just to be paranoid, I've even tried rebooting the server.

The odd thing is, this server is running the same patch levels as all of our 11.0 servers, but started showing this behaviour at the end of last week even though nothing has changed !!!

Ok - slight update - I've just tried bouncing inetd again, but I couldn't get it to restart at all and had to reboot the server again.

I've got the output from "tusc -f", but I'm not overly familiar with what it's telling me - should I be looking for anything specific ??

(Thanks again for all the help !!!).

[Alex Lavrov.]

If inetd hangs, you can attach here the last lines from tusc output. Those should be the last system calls that inetd executes before it hangs.

[Paul Murray_4]

Hi Alex - the last part of the tusc output is :-

setaudid(0) ...................................................... = 0
sigsetreturn(0x7f7bcd56, 0x6211988, 1392) ........................ = 0
sigvec(SIGHUP, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGINT, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGQUIT, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGILL, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGTRAP, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGABRT, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGEMT, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGFPE, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGKILL, 0x7f7e0820, 0x7f7e082c) .......................... ERR#22 EINVAL
sigvec(SIGBUS, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGSEGV, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGSYS, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGPIPE, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGALRM, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGTERM, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGUSR1, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGUSR2, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGCLD, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGPWR, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGVTALRM, 0x7f7e0820, 0x7f7e082c) ........................ = 0
sigvec(SIGPROF, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGIO, 0x7f7e0820, 0x7f7e082c) ............................ = 0
sigvec(SIGWINCH, 0x7f7e0820, 0x7f7e082c) ......................... = 0
sigvec(SIGSTOP, 0x7f7e0820, 0x7f7e082c) .......................... ERR#22 EINVAL
sigvec(SIGTSTP, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGCONT, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGTTIN, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGTTOU, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGURG, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGLOST, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGRESERVE, 0x7f7e0820, 0x7f7e082c) ....................... ERR#22 EINVAL
sigvec(SIGDIL, 0x7f7e0820, 0x7f7e082c) ........................... ERR#22 EINVAL
sigvec(SIGXCPU, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGXFSZ, 0x7f7e0820, 0x7f7e082c) .......................... = 0
sigvec(SIGCANCEL, 0x7f7e0820, 0x7f7e082c) ........................ ERR#22 EINVAL
sigvec(SIGGFAULT, 0x7f7e0820, 0x7f7e082c) ........................ ERR#22 EINVAL
sigvec(SIGRT1, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGRT2, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGRT3, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGRT4, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGRT5, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGRT6, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGRT7, 0x7f7e0820, 0x7f7e082c) ........................... = 0
sigvec(SIGRT8, 0x7f7e0820, 0x7f7e082c) ........................... = 0
semget(0x1090522, 1, IPC_CREAT|IPC_EXCL|0644) .................... ERR#17 EEXIST
semget(0x1090522, 1, 0) .......................................... = 325
semctl(325, 0, GETPID, 0) ........................................ = 0
semget(0x1090522, 1, 0) .......................................... = 325
semctl(325, 0, GETPID, 0) ........................................ = 0
( Terminating process 7742 ("/usr/sbin/inetd -l") )

[Alex Lavrov.]

Can you attach the file with all the output?

I see that you have some problems with getting the semaphores ....

[Paul Murray_4]

Hi Alex - attached is the full tusc output as requested.

Thanks,
Paul.

[rick jones]

Unless it is poor wording on the part of tusc, I would not have expected to see a message about the process terminating - at least not if you were following forks in the tusc with -f. From startup inetd will fork at least once to disconnect itself from the controlling terminal and all that.

Might want to add the "display pid" option for tusc, and perhaps add timestamps to see where it may be spending lots of time.

[Stephen Keane]

What does ipcs -sa give you?

[Alex Lavrov.]

Hmm, this part looks weird:

mmap(NULL, 86016, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0x5000) = 0xc0222000
mmap(NULL, 20480, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE|MAP_SHLIB, 3, 0x1a000) = 0x7f797000
close(3) ......................................................... = 0
stat("/usr/lib/libxti.2", 0x7f7e2c70) ............................ = 0
open("/usr/lib/libxti.2", O_RDONLY, 0) ........................... = 3
fstat(3, 0x7f7e2d38) ............................................. = 0
read(3, "0210010e0512@ \0\0\0\0\0\0\0\0\0".., 128) ............... = 128
lseek(3, 128, SEEK_SET) .......................................... = 128
read(3, "10\0\004\0\0\0( \001D bc\0\010\0".., 48) ................ = 48
read(3, "80\0\0\v\0\0\004\0\0\0\0", 12) .......................... = 12
lseek(3, 20480, SEEK_SET) ........................................ = 20480
read(3, "058cy 10\0\0\0a8\0\0\0p \0\0\001".., 112) ............... = 112
mmap(NULL, 86016, PROT_READ|PROT_EXEC, MAP_SHARED|MAP_SHLIB, 3, 0x5000) ERR#12 ENOMEM
close(3) ......................................................... = 0



Why the first mmap call goes OK, but then, mmap with exactly the same parameters fails with ENOMEM?

[Kent Ostby]

Paul -- I don't see a specific tie in to your problem, but you might want to check to see if you have the latest inetd patch which is PHNE_28312.

Best regards,

Kent M. Ostby

[Ermin Borovac]

From your tusc output it seems that, on startup, inetd tries to create semaphore with key 0x01090522. In your case semaphore was already created possibly by previous inetd invocation and never cleaned up (maybe due to kill -9 on inetd process).

You first need to clean up semaphore to allow inetd to start.

# ipcrm -S 0x01090522

Then you can try tusc'ing inetd again with the following flags (especially -k).

# tusc -o /tmp/tusc.log -ccc -f -k -l -n -p -v -T%T -b 80 -r all -w all /usr/sbin/inetd -l

Trace output will go to /tmp/tusc.log.