file system full - part 2

[Paul Wahner]

Hello again,

About a month ago I started a thread when I was repeatedly getting the "/tmp: file system full" error while trying to execute a man command. To correct the problem I extended /tmp by 200Mb. /tmp now has a total size of 600Mb. Now here I am a month later and I'm starting to get this error again.

I've attached a file with a number of outputs from my workstation. The glaring thing to me from this output is that I have a large /tmp logical volume with practically no files in it and man commands and a few other commands generate the error.

My workstation is a B180 running 10.20.

Help, I'm falling and I can't get up!
Paul Wahner

[Rita C Workman]

Would you be kind and run and attach the output of this:

du -k /tmp

Thanks,
/rcw

[Maureen Gunkel]

Paul,
I'd be real interested in what's in your lost+found directory, and maybe the others, too. You can probably delete everything in the lost+found directory, but don't delete the directory itself.
Hope this helps,
Mo

[Paul Wahner]

Rita,

The output of du -k /tmp is the same as the
du -kx /tmp output at the bottom of my attachment.

Paul

[John Bolene]

I would be real curious if someone was writing to the /tmp/null file. Its time was real current.

Otherwise there is a lot of space used up by something.

It could very well be that there was a large file and someone deleted it. Deleting it does not work if it is still open by someone. It just removes the directory entry so you can't see the file anymore.

I did this with syslog one time and ended up having to reboot the box to set things right.

[A. Clay Stephenson]

Do you by any chance have a large number of processes or a number of processes with very large numbers of open files. I think what is going on it that some processes are opening a temp file then unlinking the pathname in tmp.
The space is then only returned to the filesystems when all the file descriptors associated with that pathname are closed.
Have you tried running lsof?

[Tracey]

I agree with Jhon, I think it is a file that was used/deleted but still open so the OS doesn't release the space. Have you tried the fuser command to see what/who has open files to the /tmp directory? fuser /tmp -f null would be a good start.

[Jim Moffitt_1]

Maybe this will help drill down to where there re hidden files or something. If that ll of /tmp is everything printed, then something is not right. When you are only using up 15 inodes, yet have a 61% disk utilization(315 MB used). Hope this helps:

cd /tmp
find . -type f -size +${1}000000c -exec ll {} \;

[Jim Moffitt_1]

sorry, I took that line from one of my scripts:
find /tmp -type f -size +1000000c -exec ll {} \;

This should print anything greater than 1MB.

[FRED Dennison]

Paul,

1. Is there any mountpoints under /tmp? Unlikely, but space may be hidden under a directory that is mounted to somewhere else?

2. What does "fuser -c /tmp" show? (A more flashier version is 'lsof', available from the HP Porting Centre). Whenever we experience a similar problem, it has commonly been an Application still holding onto space allocated in a mount point. Have you done any "uncompress"s or sh'd a patch lately? Are all Applications that utilise /tmp shutdown completely on a regular basis?