Create core file.

[brian_31]

Team:

I want to create a core file of an application runaway process. This is what i tried
kill -6 PID (it did not create a core but killed it)
kill -3 PID (did nothing)

Please help me to create a core file so that we would know the root cause of the runaway problem.

also an intresting thing is I am also able to find two immediate parent of the runaway process still there. This script is starting with the following options..I am confused because the PPID of the runaway is not 1 but this script mentioned below.

-ksh -c /PATH/TO/SCRIPT

Thanks
Brian

[Robert-Jan Goossens]

Hi Brian,

How about using tusc or trace.

http://hpux.cs.utah.edu/hppd/hpux/Sysadmin/tusc-7.3/

http://hpux.cs.utah.edu/hppd/hpux/Sysadmin/trace-1.6/

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x62def9beca68d511abcd0090277a778c,00.html

Hope it helps.

Robert-Jan.

[Bill Hassell]

killl -3 will indeed create a core dump:

sleep 100 &
kill -3

The reason your runaway process did not create a core file is:

1. It has enabled traps inside the program and is ignoring the kill -3 signal,

2. The process is waiting on a never-to-complete I/O task, typically networking and will core dump only when the I/O completes.

[brian_31]

Hi Again:

I know that tusc would trace the system calls but we have to go thru so escalation process to install any 3rd party s/w on the production box. As far as the signals are concerned when the user launches the application (.profile and /etc/profile the signal 3 is not ignored). Please advise.

Thanks
Brian.

[Dietmar Konermann]

There are some other potential problems prevent a process from writing a core. The process needs write permission for its current working directory. And it must not run with RUID <> EUID. See man core(4).

Best regards...
Dietmar.

[Giri Sekar.]

Hi:

Make sure you are on a good patch level. I think ksh had an issue with a "-c" option.
Do /usr/contrib/bin/show_patches |grep ksh

It will tell you what ksh cumulative patch is applied.

Thanks
Giri Sekar.

[Chris Wilshaw]

Are you sure that kill -3 did nothing? It could have created the core file in a directory that you are not expecting it to use.

If it's a process running from a terminal session, you may be able to create a core file by using the CTRL+\ key combination.

[brian_31]

I checked up the Patch and it was PHCO_26783. I did not understand the RUID<>EUID. Can someone explain. All the apps starts in the users' home dir. so i am sure it did not produce a core file.

Thanks
Brian.

[Frank Slootweg]

First try Bill's simple example with the sleep(1) command, but do it as the problem user and in the problem directory.

Re: "kill -3 PID (did nothing)": Did it *really* do nothing, or *did* it kill the process but *not* generate a core file?

[brian_31]

kill -3 did nothing. I just got my prompt back and the process was still 100% CPU. kill -6 also did not create a core file but killed the process.

Thanks
Brian.

[Dietmar Konermann]

The process only writes a core if the real user ID matches the effective user ID.

E.g. if you have an executable owned by userA with suid flag set (chmod u+s) and userB executes it, then the process has real user ID of userB, but an effective user ID of userA... and it would NOT write a core.

Best regards...
Dietmar.

[Frank Slootweg]

I think your application just catches SIGQUIT (3).

Try SIGINT (2) and see what happens. That will *not* create a core file, but let us know if it terminates the process or not.

And, like I wrote before, try Bill's example and report your findings.

[brian_31]

Okay. Our process' ids are the same so we do not have that issue. As far as kill -2 is concerned we have to try it at the next oppurtinity as this is happening on the production box only. Thanks for all the help.

Thanks
Brian.

[Bill Hassell]

You won't get a core file if ulimit -c has been set to zero. Type the command: ulimit -a. If you get an error, run the /usr/bin/sh POSIX shell and type ulimit -a

Your sysadmin may have tried to control core dumps with: ulimit -c 0 (in /etc/profile) which means no one gets a core file. Tell the sysadmin to change the ulimit command to: ulimit -Sc 0 and that way, you can change the -c value after you login.

[brian_31]

Hi:

This is what my ulimit -a reports
$ulimit -a
time(seconds) unlimited
file(blocks) unlimited
data(kbytes) 65536
stack(kbytes) 392192
memory(kbytes) unlimited
coredump(blocks)4194303

Thanks
Brian