topic Re: core keeps generating in Operating System - HP-UX

core keeps generating

sycncs — Mon, 03 Mar 2003 02:39:50 GMT

Hi Folks

Usually, / is 43% used in our server.
However, there is a core file that keeps generating and will burst the root filesystem to 100%.
I am not sure what triggered this, but the command "sar" does have the effect to generate the core file.
Not that I could find anything unusual, from glance and def processes. No backup and application running, too.

Please advice. Thanks in advance.

Re: core keeps generating

Michael Steele_2 — Mon, 03 Mar 2003 02:44:10 GMT

Lots of times its a problem in the /dev sub directory. For exmaple, /dev/rmt0mn. Use:

find /dev -type f

...to parse the /dev sub directory.

Also,

du -x / | sort -rn | head

...will list large to small files in /.

Re: core keeps generating

sycncs — Mon, 03 Mar 2003 02:50:49 GMT

find /dev type f gives me /dev/null2.
du -x / |sort -rn | head did not give me any interesting clues..
What should I do next?

Re: core keeps generating

John Poff — Mon, 03 Mar 2003 02:53:00 GMT

Hi,

Do you know where the core file is and is so do you have a copy of it? You can use the 'file' command or the 'strings' command to see which executable generated the core file. Try a 'file core' to see what you get.

JP

Re: core keeps generating

sycncs — Mon, 03 Mar 2003 03:06:34 GMT

Hi..I got the following.
Core file from 'sar'

Re: core keeps generating

John Poff — Mon, 03 Mar 2003 03:15:08 GMT

Since 'sar' is generating the core file, I'd suggest checking your patches to make sure you have all the latest sar patches as well as any other patches you might need. Also, you might check your /var/adm/sa directory and make sure everything looks normal there.

JP

Re: core keeps generating

Bill Hassell — Mon, 03 Mar 2003 03:20:07 GMT

You are probably way behind on patches. Apply any patches that relate to sar and your version of HP-UX. However, a core file is generated in the current working directory and unfortunately, the default location for root's HOME is /, the WORST possible location. Move root's HOME directory to /home/root or similar. Moving it to /root won't help since it will still be part of the / directory.

By moving root's HOME to another directory, you will make your system much more reliable and if the core file is created by sar, / will not be affected (assuming you have a standard LVM layout).

Re: core keeps generating

Steven E. Protter — Mon, 03 Mar 2003 03:22:25 GMT

Current Patch, December 2002 is a great idea.

I've attached a backgroud sar script that you can run on a mounted filesystem. Core files from that won't be in /

Don't ever run these scripts in / fs

Also, don't run these scripts until you've patched up your box.

Patch list for 11.00

PHKL_28003
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHKL_28003&context=hpux:800:11:00
PHCO_25174
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHCO_25174&context=hpux:800:11:00
PHSS_26793
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHSS_26792&context=hpux:800:11:00
PHSS_26792
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHKL_23901&context=hpux:800:11:00
PHKL_28602
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHKL_28602&context=hpux:800:11:00
PHSS_19005
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHSS_19005&context=hpux:800:11:00

Special instructions, read and follow them.

The links will let you build a patch bundle with dependencies.

They will require a system boot.

SEP

Re: core keeps generating

monasingh_1 — Mon, 03 Mar 2003 09:10:26 GMT

till the time you load the patches or find the solution, just create a directory named "core" and then at least your core file will not be created and your / will not fill up....

by the way, file core or strings should help you find some hint what is generating the core file as suggested earlier..

Re: core keeps generating

sycncs — Tue, 04 Mar 2003 03:58:15 GMT

Hi All

Thank you very much for all your valued advices.
Steven, we have more than 100 servers here, all fitted with Sept 2002 patch bundles. Strange thing is - only this particular server has this problem...?
Bill, I dun think moving the root's HOME directory will help, it will only accomodate the coredump but I need to know why it triggers off the core file.

I seemed to be at a loss.

Re: core keeps generating

Adam J Markiewicz — Tue, 04 Mar 2003 10:19:39 GMT

Hi

I don't know if it really helps, you but I'm facing strange behaviour on one HP machine.

When I login the sh is started, as specified in /etc/passwd. Then I try to start bash manualy. However sometimes it ends with core (but it is not a rule). Thats the reason why I don't vave bash in /etc/passwd.
But, when it starts doing it for the first time (heavy morning?) I'm sure it will happen again at the next try.
What I have to do then is to perform cd before starting bash. It doesn't matter where I cd to. I can also do it twice, to return to the directory I started from (of course - my home directory).
After its done I can start bash without problem.

Till now I don't understand it. Mayby you have something so mysterious also?

Good luck

Adam

Re: core keeps generating

U.SivaKumar_2 — Tue, 04 Mar 2003 11:12:49 GMT

Hi,

To prevent any problem due to root filesystem full disable the core files.

As root give this command.

#ulimit -c 0

Put the above line in some startup scripts.

regards,

U.SivaKumar

Re: core keeps generating

Steven E. Protter — Tue, 04 Mar 2003 15:31:30 GMT

Interesting in your last post.

swverify \* check the output for errors.

Maybe the September Patch set has problems or other patches do.

Could be a hardware issue messing up the system.

run cstm or mstm or in X xstm and test out those hard disks, make sure they don't have any errors.

Are you getting anything in /var/adm/crash ?

If so use the attachment to analyze the dumps and send them to HP.

SEP