topic Re: Space on disk taken by nothing...?!?!? in Operating System - HP-UX

Space on disk taken by nothing...?!?!?

Jonathan Caplette_1 — Thu, 19 Sep 2002 12:28:34 GMT

Hi guys!

I have a drive in one of my server that when I use the "bdf" command it show me that data use 17GB so the drive is full at 97%... But when I use "du -k /basedir" it show me that all the data take only 15GB.. So I've lost 2GB I don't know where...

This drive is an 18GB Disk and it is configured alone in is volume group and has only one logical volume represented by my basedir...

any idea of what append???

Thanks
Jonathan

Re: Space on disk taken by nothing...?!?!?

Paula J Frazer-Campbell — Thu, 19 Sep 2002 12:30:45 GMT

Hi

Have a look at the output of:-

diskinfo
vgdisplay
lvdisplay

Your answer shuold be there.

Hth

Paula

Re: Space on disk taken by nothing...?!?!?

Pete Randall — Thu, 19 Sep 2002 12:34:58 GMT

Jonathan,

Are you sure? If I use the following example from one of my systems:

root# bdf /var
Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol8 11829248 6211554 5448870 53% /var
root# du -sk /var
6174410 /var

You see a substantial difference between the bdf and du results but you need to remember that du reports blocks. So that's 6174410 1024K blocks, or 6322595840 bytes, which is even more than what bdf shows.

Just different tools.

Pete

Re: Space on disk taken by nothing...?!?!?

Jonathan Caplette_1 — Thu, 19 Sep 2002 12:36:04 GMT

Hi Paula,

All my space is already allowed on that drive.. I've no Free PE left...

What else should I look for in diskinfo, vgdisplay or lvdisplay???

Re: Space on disk taken by nothing...?!?!?

James R. Ferguson — Thu, 19 Sep 2002 12:41:40 GMT

Hi Jonathan:

The difference may be due to the presence of one or more 'sparse' files. 'core' files fall into this category as do some database files.

Regards!

...JRF...

Re: Space on disk taken by nothing...?!?!?

Jonathan Caplette_1 — Thu, 19 Sep 2002 12:41:44 GMT

Hi Pete..

I know for that difference.. but could it bee like a 2GB of diffenrence between those two tools??? that's weird for me... look at what I got with bdf /psoftext:
(root)system:/# bdf /psoftext
Filesystem kbytes used avail %used Mounted on
/dev/vg04/lvol1 17776640 17264425 487629 97% /psoftext

and with du -sk /psoftext:
(root)system:/# du -sk /psoftext
15140273 /psoftext

Re: Space on disk taken by nothing...?!?!?

Jonathan Caplette_1 — Thu, 19 Sep 2002 12:43:27 GMT

Hi James...

could be the answer.. I've some databases on that filesystem...

I'll check!!

Re: Space on disk taken by nothing...?!?!?

James R. Ferguson — Thu, 19 Sep 2002 12:57:46 GMT

Hi (again) Jonathan:

One way to check if a file is "sparse" is to copy it with 'cp'. A sparse file will be "expanded" when copied this way. An 'ls -l' of the sparse and the expanded file will remain the same, but a 'du' will differ significantly. The copy of the original file will be inflated.

Regards!

...JRF...

Re: Space on disk taken by nothing...?!?!?

Jonathan Caplette_1 — Thu, 19 Sep 2002 13:00:43 GMT

hum... the thing, James, is that I don't have the place anywhere on that machine that I can copy a databases that big... Damn.. How could I check it otherwise????

Re: Space on disk taken by nothing...?!?!?

James R. Ferguson — Thu, 19 Sep 2002 13:20:40 GMT

Hi (again) Jonathan:

You might get a rough idea by comparing the reported file sizes with 'du' and 'ls -l' accounting for the fact that 'du' reports in 512-character blocks. Sparse files will have a smaller actual 'du' than you would calculate based on the size noted by 'ls -l'.

Regards!

...JRF...

Re: Space on disk taken by nothing...?!?!?

Shannon Petry — Thu, 19 Sep 2002 14:38:33 GMT

The answer is pretty simple really. In OLD HFS file systems, we had to deal with a 10% tolerance that was not usable.
interestingly 10% of 18GB is almost 2GB.
With LVM, many people thought that this would go away, however the 10% is pretty much a safe rule for LVM overhead. I.E. LVM structures, cache/log, etc...
what you are reporting with the du -sk command is the file system useage in that partition. what you see in bdf is the overall disk summary including the LVM overhead.

Regards,
Shannon

Re: Space on disk taken by nothing...?!?!?

George A Bodnar — Thu, 19 Sep 2002 17:43:20 GMT

Actually the difference between the kbytes column (using bdf) and the avail+used is your overhead.

When you do a du -ks /mount you should have very close to what is showing for used in a bdf.

One thing to consider is any open file that has had its directory entry delete, but not been released yet - example an open log file is deleted but the process remains.

Only way to find these types of things is using the lsof tool (or glance) and looking for files that exist.

Alternately after your next reboot try the du and compare against used.

As for the difference between kbyte and avail+used that is where you'll see your VxFS overhead.

One final thing to consider - an 18GB drive might really be 16.5GB usable - you need to run diskinfo against the device to see how many KB are actually usuable.

Re: Space on disk taken by nothing...?!?!?

Jonathan Caplette_1 — Thu, 19 Sep 2002 17:53:25 GMT

Hi Georges!!

Thanks man... I found out that there was a process that was running for a while and not used anymore... So when I killed it I found back the space I've lost...

Thanks to all others who helped me!!
ciao!

Re: Space on disk taken by nothing...?!?!?

Rory R Hammond — Fri, 20 Sep 2002 17:06:55 GMT

Jonathan,

When a program (process) opens a file, it uses a file descriptor.

If you delete the file, and the process is still running. The space is not released until the file descriptor is closed. This is usually done by killing the process.

Commonly, I have seen new admins rm /var/adm/syslog/syslog.log inorder ot free space.

The problem is that the space will not be freed until syslogd is stopped.

"fuser" might have found your problem

for instance
fuser -u /dev/vg00/lvol4 (which is my /tmp) You would find what process have files opened in tmp. Which would hopefully give you a clue to the errant program.

When removing\reduceing logfiles. It is a good practice to do a fuser on the file to see if it is has attached process. If this is the case ZERO out the file instead of removeing it
(echo "" > badlog