Freeing disk space

[Larry Basford]

Is there a better way to free up disk space?
I had a user create a very large print job by mistake. 150MG and it filled /var/spool to 100%.
When I rm'd it it took about 15 minutes before bdv /var/spool showed it gone and no one could print untill then.
On a non HP system we had the same problem and it neveer did ,in 30 minuets, clear so we rebooted the server to clean it up.

I'm thinking >the.file.name may zero it and release the space but havn't had the opertunity to try it.

Any ideas?

[A. Clay Stephenson · Accepted solution]

Hi Larry,

No, not really but you need to understand what rm (or more accurately unlink() at the system level does). It decrements the link count to a file (inode). When that count falls to zero
the directory entry is removed. When no processes have the file open, the file is then removed. In your case, because the file was printing, an lp process had the file open and thus it could not be removed from the filesystem eventhough the directory had been removed. Had you stopped the lp process then the space would have 'come back' very quickly.

Hope this clears things up, Clay

[Larry Basford]

I actually did a mv on the file to /tmp to keep it. The file moved OK but space didn't come free.

You say if I lpshut then rm it should free the space ?

[Patrick Wallek]

If you do an lpshut and then rm the file, the file should be removed and the space should become available. That is assuming that lp is the only process that has the file open. If the user still has the file open, then you will still have the problem until the user closes the file. If you want to check who has a file open, lsof is a good utiltiy to use.

Even with a move (which is essentially a cp and then a rm) the file won't be removed until the file is closed by whatever has it open.

[Charles McCary]

Larry,

a quicker way to handle for any file that doesn't "release" it's space after being removed is to do an fuser on the file. This will give you the pid(s) of the file that has the file open, you can then ps on those pids and decide whether to kill them, restart something (such as lp), etc...

[Larry Basford]

Thanks all who responded.
But if I do an mv which is a cp and a rm
and the file is on a different filesystem
and no longer ls's in the old location
bdf on /var/spool is still at 100%
and the spooler is not running.
What do I run fuser on ?
Or lsof what ?
If someone has the file open why did the file name go away.

[Charles McCary]

Hey - just touch the file name in the original directory, then do an fuser on the filename, I think this will work.

If that doesn't work, do an fuser * from within the original dir.

[A. Clay Stephenson]

Hi Larry,

You packed a lot of questions in. You will note that I did not say do an lpshut in my original response because that may not be enough. If any spooler program or filter had the original file open in /var/spool then that would hold the space until the file is closed.
Not all lp related files and especially 'homegrown' filters respond to an lpshut command.

I would need to experiment with lsof to see what it does in the case that a file has been unlinked but is help open by some process.

A very common routine in a C program is to do this to create a temporary file:

fdes = open("myfile",O_RDWR | O_CREAT,0600);
unlink("myfile");

This has the effect of open and creating "myfile" and then removing "myfile". It's really still there but there is no directory entry. This program can read and write to this non-existant file to its heart's content but the rest of the system knows nothing about it except that filesystem space is used. Only when a close(fdes) is done is the space returned. The very same thing occurred in your case. The mv did nothing to free up space in /var until the process closed the file.

Hope this clears it up a bit, Clay

[A. Clay Stephenson]

Hi Larry,

I took about 2 minutes and created a real version of the program with a two minute sleep.

In the case of an open unlinked file lsof does indicate the file but rather than the filename it displays the filesystem device. e.g. /dev/vg00/lvol6.

Clay

[Jim Turner]

BTW, if you want to free the space while one or more procs still have the file open, just redirect nothing to it like so:
> /var/spool/bad_file

Redirecting a cat of /dev/null does the same thing if you just like more typing:
cat /dev/null > /var/spool/bad_file

Of course, the original file contents will be lost, but in what are you interested at crunch time -- the file contents or the disk space?

[Bill Hassell]

/var (as you've probably seen is a very crucial filesystem for every user on the system (including root) and usually needs to be segmented to prevent mistakes in one subsystem from taking the entire filesystem down.

For a print job, I take the approach that these are not as important as loss of email, loss of system logging, and other application crashes which will occur wehn /var fills...so I would either:

lpcancel XXX

(and that removes the print job)

or cp the two files (in case someone wants to print it later) to some other filesystem like /tmp (note: 2 files, the data file AND the control file).

But to keep /var under control, consider adding a 4 Gbyte disk (they are really cheap) and creating mountpoints for:

/var/tmp
/var/spool
/var/mail
/var/adm
/var/adm/sw

Then go into single user mode (need to have /var unmounted) and temporarily mount each of the new lvols so you can move the files and directories from /var to their appropriate locations on the new disk.

Then umount the temp locations, edit fstab and verify that a mount command for each new mountpoint works OK. Naturally, you would want to do a back before any of this.

Now, if spooling fills, mail runs and logs are OK and applications run OK, etc. Same thing for email filling /var/mail...