killing process released disk space

[Shivkumar]

Our one of the application file system is getting filled up. We are running a weblogic server process which runs
java process. We had killed the weblogic server process without killing its related java process. These java process
was running. We again restarted the
weblogic server process which started different new java processes. Since then we realized our /opt/rose/ file system is getting filled very quickly.

We killed the earlier old java process and it released the disk space.
Can anyone explain why this unused java process was consuming so much disk space in the /opt/rose file system. And how killing this old (earlier) java process released disk space ?

[Patrick Wallek · Accepted solution]

This is classic Unix behavior.

The process you killed had a file open. That file was, at some point, unlinked. But since the process still had the file open and in use the space was not released. When you killed the process, the file was released and no longer in use, thus allowing HP-UX to finish its clean up and reclaim the disk space.

[A. Clay Stephenson]

You need to read the man 2 unlink man page. It it very common in UNIX to create a temp file by opening a file and then immediately unlinking it. The unlink system call remove the directory entry but does not actually free the space until 1) the file is closed or 2) the process terminates. This is exactly the mechanism the UNIX tmpfile() function uses.

[James R. Ferguson]

Hi Shiv:

As far as killing a process and seeing previously utilized disk space returned to the system, that is common.

If a process has a file open, but the process 'unlink's the file (the underlying system call to remove the file) then any directory entry for the file is removed *but* the space currently allocated is kept available to the process until it terminates.

In fact, it is common to open a temporary file; unlink it; and continue to use it for the life of the process. Upon the termination of the process, the filesystem space used by the temporary file is returned without further programmatic intervention.

Regards!

...JRF...

[Raj D.]

Shiv ,

In addition to that , you can also check if any big core file is getting generated there. Java many times creates large core files , in case of abnormal termination.
We gets many times upto 500MB core file ,from Tivoli LCF daemon and Java.

# find . -name core -print
# strings core | more
( To ascertain about the core file.)

hope this will help,

hth,
Raj.

[Arunvijai_4]

Hi Shiv,

As gurus mentioned its a classic UNIX behaviour. The proper way to shutdown weblogic by using shutdown script, this script will take care of shutting down all the "java" processes that are created when startup script. Otherwise, one more of way doing this is,

1) Kill all Java processes related with Weblogic

2) Kill the startup script.

-Arun

[Yogeeraj_1]

hi,

allow me also to add that you should configure ou system such that this case does not because a cause of downtime in the future.

We had a case recently where an astray process cause a file system to reach 100% usage and prevented a datafile to extend thus preventing our corporate ERP from functioning properly!

hope this helps too!

kind regards
yogeeraj

[Yogeeraj_1]

hi again,

again ensure that your patches are up to date such that such problem does not crop up again.

kind regards
yogeeraj