core keeps generating

[sycncs]

Hi Folks

Usually, / is 43% used in our server.
However, there is a core file that keeps generating and will burst the root filesystem to 100%.
I am not sure what triggered this, but the command "sar" does have the effect to generate the core file.
Not that I could find anything unusual, from glance and def processes. No backup and application running, too.

Please advice. Thanks in advance.

[Michael Steele_2]

Lots of times its a problem in the /dev sub directory. For exmaple, /dev/rmt0mn. Use:

find /dev -type f

...to parse the /dev sub directory.

Also,

du -x / | sort -rn | head

...will list large to small files in /.

[sycncs]

find /dev type f gives me /dev/null2.
du -x / |sort -rn | head did not give me any interesting clues..
What should I do next?

[John Poff]

Hi,

Do you know where the core file is and is so do you have a copy of it? You can use the 'file' command or the 'strings' command to see which executable generated the core file. Try a 'file core' to see what you get.

JP

[sycncs]

Hi..I got the following.
Core file from 'sar'

[John Poff]

Since 'sar' is generating the core file, I'd suggest checking your patches to make sure you have all the latest sar patches as well as any other patches you might need. Also, you might check your /var/adm/sa directory and make sure everything looks normal there.

JP

[Bill Hassell]

You are probably way behind on patches. Apply any patches that relate to sar and your version of HP-UX. However, a core file is generated in the current working directory and unfortunately, the default location for root's HOME is /, the WORST possible location. Move root's HOME directory to /home/root or similar. Moving it to /root won't help since it will still be part of the / directory.

By moving root's HOME to another directory, you will make your system much more reliable and if the core file is created by sar, / will not be affected (assuming you have a standard LVM layout).

[Steven E. Protter]

Current Patch, December 2002 is a great idea.

I've attached a backgroud sar script that you can run on a mounted filesystem. Core files from that won't be in /

Don't ever run these scripts in / fs

Also, don't run these scripts until you've patched up your box.

Patch list for 11.00

PHKL_28003
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHKL_28003&context=hpux:800:11:00
PHCO_25174
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHCO_25174&context=hpux:800:11:00
PHSS_26793
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHSS_26792&context=hpux:800:11:00
PHSS_26792
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHKL_23901&context=hpux:800:11:00
PHKL_28602
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHKL_28602&context=hpux:800:11:00
PHSS_19005
http://www1.itrc.hp.com/service/patch/patchDetail.do?patchid=PHSS_19005&context=hpux:800:11:00

Special instructions, read and follow them.

The links will let you build a patch bundle with dependencies.

They will require a system boot.

SEP

[monasingh_1]

till the time you load the patches or find the solution, just create a directory named "core" and then at least your core file will not be created and your / will not fill up....

by the way, file core or strings should help you find some hint what is generating the core file as suggested earlier..

[sycncs]

Hi All

Thank you very much for all your valued advices.
Steven, we have more than 100 servers here, all fitted with Sept 2002 patch bundles. Strange thing is - only this particular server has this problem...?
Bill, I dun think moving the root's HOME directory will help, it will only accomodate the coredump but I need to know why it triggers off the core file.

I seemed to be at a loss.

[Adam J Markiewicz]

Hi

I don't know if it really helps, you but I'm facing strange behaviour on one HP machine.

When I login the sh is started, as specified in /etc/passwd. Then I try to start bash manualy. However sometimes it ends with core (but it is not a rule). Thats the reason why I don't vave bash in /etc/passwd.
But, when it starts doing it for the first time (heavy morning?) I'm sure it will happen again at the next try.
What I have to do then is to perform cd before starting bash. It doesn't matter where I cd to. I can also do it twice, to return to the directory I started from (of course - my home directory).
After its done I can start bash without problem.

Till now I don't understand it. Mayby you have something so mysterious also?

Good luck

Adam

[U.SivaKumar_2]

Hi,

To prevent any problem due to root filesystem full disable the core files.

As root give this command.

#ulimit -c 0

Put the above line in some startup scripts.


regards,

U.SivaKumar

[Steven E. Protter]

Interesting in your last post.

swverify \* check the output for errors.

Maybe the September Patch set has problems or other patches do.

Could be a hardware issue messing up the system.

run cstm or mstm or in X xstm and test out those hard disks, make sure they don't have any errors.

Are you getting anything in /var/adm/crash ?

If so use the attachment to analyze the dumps and send them to HP.

SEP