topic Re: Difference between du and bdf - bad process eats disk space in Operating System - HP-UX

Difference between du and bdf - bad process eats disk space

Karel Charousek — Thu, 03 May 2007 04:07:10 GMT

Hello,

bdf shows 9GB is used out of 10GB on /var
but du -sk /var shows 4GB is used.

How can I find the bad process and the way it is eating the disk space in general?

(I've found that it should be bad process because
when I shut down oracle processes the space was released. Filesystem /var is used
only for log of oracle processes (no datafiles and no database logs))

Regards

Karel

Re: Difference between du and bdf - bad process eats disk space

Wouter Jagers — Thu, 03 May 2007 04:38:47 GMT

I've seen this happen before when people (re)moved oracle logfiles while the database is running.

When a file is removed while a process still has it open, the space used by the file will not be released until the process releases it. That's why restarting the database usually helps.

In short: removing files while they are held by processes is tricky ;-)

Cheers,
Wout

Re: Difference between du and bdf - bad process eats disk space

John Waller — Thu, 03 May 2007 05:09:01 GMT

Hi,

You can use lsof to identify the processes which have open files on a particular filesystem. I normally use "lsof +d " command then look at the SIZE/OFF column for large files. You can get lsof from http://www.mirrors.wiretapped.net/security/host-security/lsof/binaries/hpux/

Re: Difference between du and bdf - bad process eats disk space

Wouter Jagers — Thu, 03 May 2007 05:15:47 GMT

Adding to that, the command "lsof +aL1 /var" (without the quotes) should display these 'unlinked open files', if I remember correctly.

Cheers

Re: Difference between du and bdf - bad process eats disk space

Karel Charousek — Thu, 03 May 2007 05:42:17 GMT

Hi,

the thing with removing open files is known to me and I didn't remember that I done something similar.

I've snapped the output of lsof.
Unfortunately I didn't snapped ps -ef.
There is following suspected line ...
oracle 29364 oracle 11u REG 64,0x8 6397549837 113293 /var (/dev/vg00/lvol8)
Column SIZE shows 6.3GB
Does that mean that there was unnamed unlinked file of size 6.3GB?

I've looked at the man page of lsof..
lsof +aL1 /var ... it is the right command

Regards

Karel

Re: Difference between du and bdf - bad process eats disk space

Wouter Jagers — Thu, 03 May 2007 07:08:00 GMT

This is from the FAQ of lsof and applies to HP-UX:

" Since the size of the kernel name cache is limited and the
cache is in constant flux, it does not always contain the
names of all components in an open file's path; sometimes
it contains none of them.

Lsof reports the file system directory name and whatever
components of the file's path it finds in the cache, starting
with the last component and working backwards through the
directories that contain it. If lsof finds no path
components, lsof reports the file system device name instead."

( section 3.1 from http://www.reznor.com/tools/lsof/FAQ )

..I guess that's why you're just getting the device name :-/

Cheers,
Wout

Re: Difference between du and bdf - bad process eats disk space

Karel Charousek — Thu, 03 May 2007 07:24:10 GMT

Hi,

Now I've possibility to find unlinked files on a filesystem with lsof +aL1 and to identify the process.

Thank you for all answers.

Regards

Karel

Re: Difference between du and bdf - bad process eats disk space

Karel Charousek — Thu, 03 May 2007 07:24:41 GMT

Re: Difference between du and bdf - bad process eats disk space

Wouter Jagers — Thu, 03 May 2007 07:48:19 GMT

No problem :-)

Thanks for also checking out http://forums1.itrc.hp.com/service/forums/helptips.do?#33 on how to reward points to useful answers.

Welcome to the forums !

Cheers,
Wout