topic Re: /usr file system issue in Operating System - HP-UX

/usr file system issue

vthiru — Wed, 14 Jan 2009 05:22:35 GMT

Hi Admins,

In one of our HP-UX 11.11 server, the /usr file system is at 97%. We found one file of size 550MB and we deleted that file. But even after deleting the file, the file system didn't get reduced. Still it is at 97%. Please let me know is there a way to find why the file system didn't get reduced.

Thanks in Advance.

Thiru

Re: /usr file system issue

Avinash20 — Wed, 14 Jan 2009 05:58:15 GMT

This could be due to unlinked files which might has not got deleted.

could you compare the output of

# bdf
# du -sk /usr

Compare the avail section in bdf and the above output

Re: /usr file system issue

vthiru — Wed, 14 Jan 2009 06:06:23 GMT

Thanks Avinash, but there are lot of file system like below:

/usr
/usr/local/fsa
/usr/local/fsb
/usr/local/fsc
/usr/local/fsd
/usr/fse

So the output doesn't help really.

servera / > bdf /usr
duFilesystem kbytes used avail %used Mounted on
/dev/vg00/lvol1 2138112 2040726 91331 96% /usr

servera / > du -ks /usr
9543016 /usr

Any help will be appreciated.

Thanks
Thiru

Re: /usr file system issue

Avinash20 — Wed, 14 Jan 2009 06:14:03 GMT

du -xk /usr

Check the last value

Re: /usr file system issue

Avinash20 — Wed, 14 Jan 2009 06:18:29 GMT

There may be remarkable differences in the bdf(1) and du(1) outputs, because bdf(1) looks at file system statistics, while du(1) looks at the files it finds in the directories.

Under certain circumstances, files that actually exist in a file system, might not be visible. There are two reasons for that, which are also the reason why du(1) wouldn't count their sizes when calculating the disk usage:

Files that have already been deleted, but are still opened by running processes
You can find such files with
crashinfo -unlinked

It will print their inodes and sizes, and also the processes that still have this file open, e.g.

PID PPID COMMAND INODE DEVICE SIZE(bytes)
3844 1 swagentd 6725 /dev/vg00/lvol6 133

These files will still occupy space in the file system, but they will not be visible in the directory listings because their directory entries have already been deleted. The files themselves will only be deleted after all processes that have the file open either close the file, or terminate.

If that doesn't happen on its own, you might need to kill the appropriate processes, to free the disk space.

Re: /usr file system issue

vthiru — Wed, 14 Jan 2009 06:24:46 GMT

Thanks again Avinash, please find the du -xk /usr

1473748 /usr

but i am not able to find crashinfo in my server. Please let me know from where can i get crashinfo.

Thanks,
Thiru

Re: /usr file system issue

Avinash20 — Wed, 14 Jan 2009 06:27:35 GMT

you need to get the crashinfo script from HP.
They will provide you the script on request.

Re: /usr file system issue

Dennis Handly — Wed, 14 Jan 2009 06:31:56 GMT

>We found one file of size 550MB and we deleted that file.

Do you remember the filename?
You may need to run lsof to see who has it open.

Re: /usr file system issue

Avinash20 — Wed, 14 Jan 2009 06:36:08 GMT

servera / > bdf /usr
duFilesystem kbytes used avail %used Mounted on
/dev/vg00/lvol1 2138112 2040726 91331 96% /usr

Used: 1993 MB

du -xk /usr
1473748 /usr

Actual Used: 1439 MB

So the difference is around 500MB.
## You could find the unlinked file via

# crashinfo -unlinked

You could obtain the script from HP Solution Center

Re: /usr file system issue

vthiru — Wed, 14 Jan 2009 06:46:12 GMT

Thanks Avinash/Dennis. There is no crashinfo and lsof installed in the server. But i could see a process that is running and i suspect that it could hold the file. Let me work with the application team and find out whether they will be able to stop and start the process once and see if that helps.

Keeps you people posted.

Thanks again,
Thiru

Re: /usr file system issue

vthiru — Wed, 14 Jan 2009 06:50:44 GMT

Thanks Dennis/Avinash,

Once the application team stopped the process, the file system usage got reduced.

servera / > bdf /usr
Filesystem kbytes used avail %used Mounted on
/dev/vg00/lvol1 2138112 1579551 523683 75% /usr

Thanks for the help.

Thiru

Re: /usr file system issue

Avinash20 — Wed, 14 Jan 2009 06:51:13 GMT

Yes, if you know the process, you could try to stop or kill it with -9

But before killing that process, please dicuss this with your application team.