Newbie: constantly running out of space on /

[Scott Sabo]

For some reaons, I am ALWAYS running out of space on / . I cannot even run SAM at this point, nor can users log in.

What the heck! How do I figure out what is on /?

This is from BDF:

Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol3 204800 204792 8 100% /

Thanks in advance.
Scott

[Jeff Schussele · Accepted solution]

Hi Scott,

First step is to determine just what/where the big files are - run

du -akx / | sort -nr | more

This sorts 'em with the biggest hitters at the top & restricts the scan to the / filesystem.
I suspect something or some one may be writing large temp files to /tmp which is a *big* no-no. Only the OS should be using /tmp, users & processes are supposed to use /var/tmp.

HTH,
Jeff

[Mark Greene_1]

Are /home, /tmp, and /var seperate file systems? If not, look there for large files or large numbers of small files being created very quickly. Look especially in /var/tmp for files to remove.

You may also have a process writing to disk to a file that has been removed. When you rm a file and there's a process still writing to it, the disk space isn't given back until the process dies.

[Scott Sabo]

Aargh.

It looks like while I was out on vaca, the temp admin decided to put a huge tar file in a directory in /.

Now, I'm a newbie, but this guy is a no-bie.

Thans for the help and the cool commands .. I like the sort by size.

[Kent Ostby]

Scott --

Two things:

ll /

What DIRECTORIES do you have listed here and which ones (see bdf) are NOT mounted.

The only directories that are not mounted here should be /dev, /root /sbin /etc and perhaps /net.

Other directories listed should show up as mounted (i.e. /opt /var /tmp /usr /home)

Secondly:

cd /
find . -xdev -size +1000000c

This will find files on that device (xdev) that are > 1 meg in size.

Be CAREFUL in deleting things.

Let us know what you find and we may be able to tell you more about why the problem keeps happening.

[Steven E. Protter]

Its possible that files on root are hidden.

umount /home
copy a bunch of files in there.

mount /home

You don't see any of the files because they are on root fs. But they are there.

If thats happened in /var or /tmp you'll probably want to boot to single use mode and clean up things that way.

/ fs should really have no files on it. It has folders for mounting and device drivers.

A common problem also is acciental file create in /dev/rmt filesystem. Somone botches a transfer to tape and copies the files to /dev/rmt There should be no normal files there.

SEP

[Prashant Zanwar_4]

find / \( -name *.txt -o -name *.log -o -name *.pdf -o -name core \) -xdev -exec ls -altr {} \;

Above will help you to figure out what file you can get rid off under root.
Hope it helps
Thanks
Prashant

[Steve Post]

I apologize if this is insulting. But I had this happened to me once. Maybe this is happening here.

I have a box. I run "bdf". I get the mount points and the % usage. Let's say I have the standard hpux mount points, plus:
/mountA
/bigdiskB
/home/workarea.

Now someone comes along and dumps a pile of files and directories to /bills_stuff. There IS no /bills_stuff mount point. So this goes to the parent directory that IS a mount point.....root. That / mount point can't hold all of bills_stuff. So you get your filesystem full problem.

Maybe you have some software that was grabbed from another box? Maybe it is using $VAR2343/bills_stuff and $VAR2343 is not defined? So it goes to /bills_stuff? Ah ha. Same thing, but done by a program instead of a human.

I hope I'm WRONG here.
steve

[Prashant Zanwar_4]

Hi,
If it is cluster system it may happen so that you still habe mount points under root and some one may keep copying files under this mount point although FS is not mounted.
Like DBA may copy some control files under it thinking it's his area and in actual FS is not mounted as the package is on the other node.
Hope it helps
Prashant

[Scott Sabo]

I think it was the big tar file that someone dumped, but it may have been because of what Steve refered to when he talked about people copying to a non-existant mount point, with the file going to / by default.

I'm going to keep a close eye on the space and see where we stand.

As always, thanks for your help. I've copied down some of the commands listed here and will use them for checking on file sizes.

Scott

[Scott Sabo]

Kent-

When you ask "What DIRECTORIES do you have listed here and which ones (see bdf) are NOT mounted."...

I can tell you that I have mount points here for some fs that have been exported on another system. Is that NOT a good practice to do that from here (here being /).

If I do the "ll /" I see the mount points for the exported fs from the other box.

Thanks,

Scott