topic Re: user id processes not get killed in Operating System - HP-UX

user id processes not get killed

Abhilash Krishnan — Thu, 12 Mar 2009 07:05:19 GMT

Hi all
we are using ssh for all of our servers.
their is one problem we have no users login in server but still it shows some user in who output.When i kill that process it is not getting killed.So user not able to login because we have the policy that user can login only 3 times.Please see output below and help me .Its urgent.

smsc2@root:/>who -u
amogh pts/0 Feb 3 14:43 old 16316 172.16.18.184
amogh pts/3 Jan 30 16:27 old 5956 172.16.202.154
srikanth pts/2 Mar 12 09:58 . 20506 172.16.18.184

smsc2@root:/>kill -9 16316
kill: 16316: The specified process does not exist.
smsc2@root:/>kill -9 5956
kill: 5956: The specified process does not exist.
You have mail in /var/mail/root

Re: user id processes not get killed

john123 — Thu, 12 Mar 2009 07:10:30 GMT

ohh.. looks like they are defunct process.
What does ps -ef |grep 16316 shows..?

Regards
john

Re: user id processes not get killed

Abhilash Krishnan — Thu, 12 Mar 2009 07:12:42 GMT

It doestn't show any process.

Re: user id processes not get killed

Avinash20 — Thu, 12 Mar 2009 07:25:20 GMT

>/var/adm/btmp
>/var/adm/wtmp

After some time

# who -u

Re: user id processes not get killed

Ganesan R — Thu, 12 Mar 2009 07:36:37 GMT

process could be zombie which you cannot kill it. Zombie process will not consume any resources other than entry in process table.

Re: user id processes not get killed

Abhilash Krishnan — Thu, 12 Mar 2009 08:10:43 GMT

/var/adm/btmp
sh: /var/adm/btmp: Execute permission denied.

it could be zombe but i have to remove that entry from utmp

Re: user id processes not get killed

Matti_Kurkela — Thu, 12 Mar 2009 08:10:59 GMT

Your /etc/utmp and/or /etc/utmpx files may have been corrupted. Older versions of HP-UX have only /etc/utmp, modern ones have /etc/utmpx too.

If all users have now logged out, you can clear the utmp and utmpx files. The command to do this is "> /etc/utmp" for the utmp file and "> /etc/utmpx" for the utmpx file.
After doing this, you should log out. At next login, the utmp file(s) will be automatically recreated.

Normally the process that is managing the user's login session will write to /etc/utmp and/or /etc/utmpx when it exits, to mark that the session has ended. When someone logs in again and gets the same pts/number, the session record is automatically recycled. But if the session is ended with "kill -9", this process may be disrupted and the incorrect session information may remain until the next reboot or until the utmp files are manually cleared.

You should use "kill -HUP" and/or just an ordinary "kill" (without -9) to remove old sessions whenever possible. Use "kill -9" only in special cases, when the gentler forms won't work. An interactive shell won't usually die with an ordinary "kill", but will die with "kill -HUP".

MK

Re: user id processes not get killed

Avinash20 — Thu, 12 Mar 2009 08:20:16 GMT

You missed ">" sign

# cd /var/adm
# cat /dev/null > wtmp
# cat /dev/null > btmp
# login

# who -u

Re: user id processes not get killed

Avinash20 — Thu, 12 Mar 2009 08:26:42 GMT

Oops.. Matti.. is correct.
I went via man page of who command

DESCRIPTION
The who command can list the user's name, terminal line, login time,
elapsed time since input activity occurred on the line, the user's
host name, and the process-ID of the command interpreter (shell) for
each current system user. It examines the "/etc/utmp" file to obtain
its information. If file is given, that file is examined. Usually,
file is /var/adm/wtmp, which contains a history of all of the logins
since the file was last created.

## you could follow Matti's plan.

Re: user id processes not get killed

Abhilash Krishnan — Thu, 12 Mar 2009 09:14:00 GMT

Hi MK thanks now who output not showing any user.Can i know when users are login how i can remove old users without affecting login users.

Re: user id processes not get killed

Avinash20 — Thu, 12 Mar 2009 09:42:39 GMT

Same way which you followed.

# who -u

If you like to remove those users

# kill -9

Re: user id processes not get killed

Avinash20 — Thu, 12 Mar 2009 09:44:36 GMT

Also

" I have assigned points to 12 of 91 responses to my questions. "

Please assign point to your thread if you get your answers.

No points for this please.

Re: user id processes not get killed

Hakki Aydin Ucar — Thu, 12 Mar 2009 10:48:14 GMT

reboot your server,

Re: user id processes not get killed

avizen9 — Thu, 12 Mar 2009 12:36:57 GMT

i had rebooted server when i got this issue before couple of month, i tried few way to kill it but failed.

you may try to reboot your server.

Re: user id processes not get killed

udayraj kotian — Fri, 13 Mar 2009 03:12:51 GMT

Is any procedure to kill it without reboot without affecting login users.

Re: user id processes not get killed

Dennis Handly — Fri, 13 Mar 2009 04:51:01 GMT

>Is any procedure to kill it without reboot without affecting login users.

You will manually have to edit the /etc/utmp and /etc/utmpx files.
You may have to stop utmpd(1M) if on 11.23.
You can use fwtmp(1M) to format the /etc/utmp file but after you restart utmpd, it says it reads utmps. So you'll have to write a program like fwtmp to work on utmps(4) formats.

>MK: But if the session is ended with "kill -9", this process may be disrupted and the incorrect session information may remain

This is a broken design and basically says this info isn't important. There should be a system demon that makes sure this works correctly. That's probably why there is now utmpd(1M).

Re: user id processes not get killed

Alexey Vekshin — Thu, 06 Aug 2009 08:02:42 GMT

Dennis Handly wrote
> You can use fwtmp(1M) to format the /etc/utmp file but after you restart utmpd, it says it reads utmps. So you'll have to write a program like fwtmp to work on utmps(4) formats.
...
> This is a broken design and basically says this info isn't important. There should be a system demon that makes sure this works correctly. That's probably why there is now utmpd(1M).

As far as I can tell from 2 days of investigation you are absolutely right on both accounts.
Reality as of 11v3 aka B.11.31 seems to be like this:
- there are utmpd + bunch of files
- /etc/utmp: good old legacy utmp
- /var/adm/wtmps: utmp log, updated (by utmpd?) too
- /var/adm/btmps: btmp log, updated (by utmpd?) too
- /etc/utmpx: newest utmp, reads deprecated (only if utmpd is not running); updated by utmpd
- /etc/utmps: utmpd saves state here on exit
- plus in-memory utmpd database, which is currently Right Way to access
- different programs use different DBs
- who uses utmpd, but 'who /etc/utmp' is ok too
- last/lastb use wtmps/btmps, last -f use /var/awm/wtmp
- utmpd is good but not fully reliable solution
- if shell is killed with -9, its tty record is forever in utmpd database
- restarting utmpd (whith /sbin/init.d/utmpd stop/start) does not help -- all is saved
in utmps and read back
- it would be good if HP added periodic check for dead PIDs of process leaders or at least
some cleanup at restart
- current crop of tools are unable to fix utmpd-related things
- fwtmp -X < /var/adm/wtmps is ok, fwtmp < /etc/utmp is ok too, but utmps and utmpx are
not accessible with fwtmp (different format)
- ./fwtmp < /etc/utmpx seems to work but resuls are total and obvious crap
- so there is no way to get rid of stale entries in utmpd DB except reload
- or maybe stopping utmpd and 'echo > /etc/utmps' before start, loosing current logins

I wrote 2 small tools to fix utmpd DB without reboot. Maybe they'll be userful to someone.

- read_utmps.c
#include
#include

int main()
{
// see /usr/include/utmps.h
struct utmps *wtmps;

// read in-memory utmpd DB
while ((wtmps = getutsent(sizeof(struct utmps))) != NULL) {
printf("%-16.16s %-8.8s %-12.12s %5ld %2hd %lu %s\n",
wtmps->ut_id, // unique ID
wtmps->ut_user,
wtmps->ut_line,
wtmps->ut_pid,
wtmps->ut_type, // 7 USER_PROCESS, 8 DEAD_PROCESS
wtmps->ut_tv.tv_sec,
wtmps->ut_host
);
}
}

- write_utmps.c
#include
#include

int main(int argc, char **argv)
{
// see /usr/include/utmps.h
struct utmps *wtmps;

if (! argv[1]) {
fprintf(stderr, "ERROR: pleas give ut_id of stale session as argument\n\n");
fprintf(stderr, "procedure is usually as follows:\n");
fprintf(stderr, "- find dead sessions like 'who -u | grep old'\n");
fprintf(stderr, " - ensure they're really dead with 'ps -xp '\n");
fprintf(stderr, "- find ut_id with read_utmps\n");
fprintf(stderr, " - 1st field, usually == or pts/\n");
fprintf(stderr, "- end session with write_utmps \n");
fprintf(stderr, " - requires root privs\n");
return(1);
}

// read in-memory utmpd DB
while ((wtmps = getutsent(sizeof(struct utmps))) != NULL) {
if (strcmp(wtmps->ut_id, argv[1]) == 0) {
printf("bingo, %s found\n", argv[1]);
printf("%-15.15s %-8.8s %-12.12s %5ld %2hd %lu %s\n",
wtmps->ut_id, // unique ID
wtmps->ut_user,
wtmps->ut_line,
wtmps->ut_pid,
wtmps->ut_type, // 7 USER_PROCESS, 8 DEAD_PROCESS
wtmps->ut_tv.tv_sec,
wtmps->ut_host
);
// mark it as DEAD to end session
if (wtmps->ut_type == USER_PROCESS) {
wtmps->ut_type = DEAD_PROCESS;
time(&wtmps->ut_tv.tv_sec);
pututsline(wtmps, sizeof(struct utmps));
}
}
}
}

Re: user id processes not get killed

Dennis Handly — Thu, 06 Aug 2009 08:59:14 GMT

>Alexey: from 2 days of investigation

I may have spent that long on it, over several questions.

>if (!argv[1]) {

You might want to do the more obvious test:
if (argc < 2) {

fprintf(stderr, "ERROR: please give ut_id of stale session as argument\n\n");

You can combine all of these fprintf into one so you don't make libc sweat:
fprintf(stderr, "ERROR: please give ut_id of stale session as argument\n\n"
"procedure is usually as follows:\n"
" - ensure they're really dead with 'ps -xp '\n"
"- find ut_id with read_utmps\n"
" - 1st field, usually == or pts/\n"
"- end session with write_utmps \n"
" - requires root privs\n");

Re: user id processes not get killed

Alexey Vekshin — Tue, 11 Aug 2009 06:16:09 GMT

Dennis Handly wrote

> >if (!argv[1]) {
> You might want to do the more obvious test: if (argc < 2) {

I rarely write C -- only then there is no ready-made perl module for a task :)

Style aside, there is definitely room for improvement in write_utmps --
iterating whole DB is unnesessary because getutsid/getutsline allows us to
fetch session record by ID. Those 2 utils are quick hacks not intended for
continuous industrial use but to fill the void and maybe spare someone a
reboot or two.

Re: user id processes not get killed

R Bray — Wed, 10 Mar 2010 17:00:15 GMT

Thanks for the C template! This allowed us to mark "DEAD" a corrupted entry in utmpx that was causing issues in our application, simply replacing the header with utmpx.h and replacing utmps functions with utmpx functions.