Operating System - HP-UX
1834814 Members
2869 Online
110070 Solutions
New Discussion

Full partition, but nothing in it

 
Kim Kendall
Regular Advisor

Full partition, but nothing in it

We have a /tmp partition that is 2GB. Last night, it hit 100% full. Someone went in and removed some files.

Later, it hit 100% full again. I went in this time and anything that didn't have a current date stamp was removed. What was left amounted to 1-2MB.

The %used (from bdf command), never seemed to go down. And sure enough, /tmp filled up again. But there are no files or directories that can account for the 2GB.

My hunch is that the first time someone removed files from /tmp, they ended up removing something that was open for write, and the system never gave up that space in memory, and that may be hanging onto (allocated) disk space.

Any other ideas of why? And any ideas how to find the villan?
13 REPLIES 13
Pete Randall
Outstanding Contributor

Re: Full partition, but nothing in it

Kim,

Probably the only way to find the villain is to use lsof.

Pete

Pete
Ken Hubnik_2
Honored Contributor

Re: Full partition, but nothing in it

Sounds like the process that was the villiam was still running and even if you deleted the file until the process goes away the space will not show up.
T G Manikandan
Honored Contributor

Re: Full partition, but nothing in it

The file system /tmp is used for all temporary work area of all the processes.
Probably when the system is rebooted the /tmp is cleared.

Also,

Now /tmp is full is that some process is filling up the fil e system.
you need to identify the process that is causing this.

you can use lsof for this.
Check this link for the download site for lsof.

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0x346d402f24d5d61190050090279cd0f9,00.html

shutdown this process and then start.

That should clear up the space
Karthik S S
Honored Contributor

Re: Full partition, but nothing in it

Try this command,

fuser -ck /tmp

and check for larger files again under that partition ..

Regards,
Karthik
For a list of all the ways technology has failed to improve the quality of life, please press three. - Alice Kahn
T G Manikandan
Honored Contributor

Re: Full partition, but nothing in it

steven has a suggestion on to find the number of open files by a process

http://forums.itrc.hp.com/cm/QuestionAnswer/1,,0xfa2a8f960573d611abdb0090277a778c,00.html

check it
Stanimir
Trusted Contributor

Re: Full partition, but nothing in it

Hi!
I have such problem months ago, and
#fsck -p
was my solution.
Regards, Stan
David_246
Trusted Contributor

Re: Full partition, but nothing in it

Hi,

At first I would like to strongly un-recommend you to do the "fuser -ck /tmp" !!

This will kill all open processes in /tmp, i.e. it will kill your system.
The best you can do is a "fuser -cu /tmp", This will tell you the current processes open in /tmp.
The reason for your error is that someone has removed a file while a process still had the file open for writing. Therefor always first check a file before deleting it, escpecialy when residing in /tmp. You can do this using "fuser -fu /tmp/".

Ok, what to do :
- bdf /tmp
- cd /tmp; du -ks .

These two should differ a lot (df counts, bdf looks up free space).
If so, do the "fuser -cu /tmp" to see all the current open processes in /tmp. ps -ef | grep all of them to see if this could be your file. When killing the right process bdf immediatly shows you the correct free space.
Of course you can compare this again with "df-ks .".


Never delete a file in /tmp without checking the file first with "fuser -fu /tmp/
And if the file is in use, but has to be deleted, then always use : "> /tmp/"

This will only zero the file and will not change the inode, etc.


Good luck !!


Regs David
@yourservice
Kim Kendall
Regular Advisor

Re: Full partition, but nothing in it


>> And if the file is in use, but has to be deleted, then always use : "> /tmp/"

Yeah, I wish he would have done this, or cat'd /dev/null into it.

I already used du to to see what was in tmp as a whole. There's nothing in there besides a few small files.

tried lsof /tmp and the output doesn't give me the files that are open. Just the dir. (see attached)
Kim Kendall
Regular Advisor

Re: Full partition, but nothing in it

Stanimir -

What kind of system do you use that you can use a -p option with fsck?
Ravi_8
Honored Contributor

Re: Full partition, but nothing in it

Hi,

#umount /tmp
#fsck -o full /dev/vg00/lvol4

may solve problem
never give up
Kim Kendall
Regular Advisor

Re: Full partition, but nothing in it

You can't unmount /tmp (unless you're prepared to go to single user)

Also tried lsof -s +D /tmp
(see attached)

Nothing that accounts for a full 2GB fils system.

Vincent Fleming
Honored Contributor

Re: Full partition, but nothing in it

From some of your attachments, it may be oracle that has a file open. Can you stop and restart the database?
No matter where you go, there you are.
Kim Kendall
Regular Advisor

Re: Full partition, but nothing in it

Yeah, that was my guess as well.