topic Re: How to identify any processes running on a huge directory structure? in Operating System - HP-UX

How to identify any processes running on a huge directory structure?

Steve Post — Thu, 03 Mar 2011 18:19:23 GMT

I have a big directory structure with lots of junk in it. I want to make sure it is not being used before I delete it. Is there a utility, or should I run fuser FILENAME, 259,432,343 times?

Re: How to identify any processes running on a huge directory structure?

Torsten. — Thu, 03 Mar 2011 18:23:49 GMT

IMHO the tool lsof is better than fuser here.

But it is still hard to decide.
What if the files are not currently open, but an application will need it later?

Whatever you will do, backup the files before delete them.

Re: How to identify any processes running on a huge directory structure?

James R. Ferguson — Thu, 03 Mar 2011 18:27:29 GMT

Hi Steve:

You don't say, but if the directory is mounted, you won't be able to unmount it until there are no open files and/or no one is 'cd'ed into it.

Regards!

...JRF...

Re: How to identify any processes running on a huge directory structure?

Steve Post — Thu, 03 Mar 2011 18:41:42 GMT

"IMHO the tool lsof is better than fuser here." I don't want to lsof 259,432,343 times either.

The directory I am talking about is not a filesystem. It is a sub,sub,sub-directory of a filesystem. It is an old copy of perl.

Maybe I could do this in reverse? Is there a command that identifies files in use given a list of processes? I know that I have a small set of apache test websites that might be using this subdirectory. I'd rather run a command 30 times instead of, well, a LOT of times.

On a side note, the idea of shutting down the computer will make sure the files are not in use. WELL....That is until I bring it up again, then some unknown application would go nuts trying to use files I trashed. THAT's no good.

Re: How to identify any processes running on a huge directory structure?

Matti_Kurkela — Thu, 03 Mar 2011 19:24:41 GMT

In Unix, there is almost always more than one way to do things. Here are a few that came to my mind for a case like this:

- A single "fuser -c " will list you all processes running on the filesystem. That is likely to be much less than 259 432 343 processes. Then "lsof -p " can be used to tell what each process is accessing, unless you happen to know it by some other way (e.g. by recognizing the process and knowing what it *does* use). If that is still too long a list, then pipe it to "grep sub-sub-sub-directory" to see at once whether it uses your directory structure or not.

- lsof +D can do the job at once, although the man page warns it might be slow and take quite a bit of memory to run. If it outputs anything at all, you'll know something is using one of the files within the directory structure.

- If you choose to use fuser, you probably will need to run fuser a considerably smaller number of times if you put multiple filenames on each line. And you certainly don't have to type any of those lines yourself. Instead, you should type something like:

find -exec fuser \+

...and let the machine do what it can do best.

MK

Re: How to identify any processes running on a huge directory structure?

Steve Post — Thu, 03 Mar 2011 19:33:51 GMT

It looks like I need to get tutorials on lssf.

...and need to give Torsten a few more points.

Re: How to identify any processes running on a huge directory structure?

Steven Schweda — Thu, 03 Mar 2011 19:42:26 GMT

> On a side note, the idea of shutting down
> the computer will make sure the files are
> not in use. WELL....That is until I bring
> it up again, then some unknown application
> would go nuts trying to use files I
> trashed. THAT's no good.

Why "trash" the files? Why not simply rename
the top-level directory (before the
shut-down), and then wait for complaints
after you bring it back up? If you get none
(within some interval deemed to be
"adequate"), _then_ you can delete ("trash")
the directory tree. If you get some, then
simply rename it back to what it was.

"Seemples".

Re: How to identify any processes running on a huge directory structure?

Steve Post — Thu, 03 Mar 2011 20:05:43 GMT

The idea of moving stuff and waiting on complaints is drastically dependent on what the complaints will be, and from who they are coming from. I have to be responsible. That might mean that I cannot allow a situation that causes complaints.

Ah. But you know that.

Re: How to identify any processes running on a huge directory structure?

Torsten. — Fri, 04 Mar 2011 11:32:23 GMT

I see a lot of problems here, but not really a simple solution ... :-(

Assuming the directory is deep under let's say /opt, everything related to the mountpoint is difficult (unmount, actions related to the mounted lvol, etc ...).

However, if you say it could be related to apache, what will likely happen?

The files are sitting around doing nothing (not opened).

If a user opens a link to a script, the script will be executed, the files opened and closed again.

In such case looking for open files will not give you the results you want to have.

IMHO there are 2 possible solutions:

- don't touch anything
- trial and error

Maybe 1) is the better solution ...

;-)

Re: How to identify any processes running on a huge directory structure?

Jose Mosquera — Fri, 04 Mar 2011 11:52:28 GMT

Hi,

Audit your mount point several times by several days to observe the behavior inside. Uses your desired tool (lsof or fuser) to do this. :)

Still, if you doubt to delete something, move it to a recycling point.

Rgds.

Re: How to identify any processes running on a huge directory structure?

Steve Post — Fri, 04 Mar 2011 13:01:13 GMT

LIST=`/usr/bin/find /A/perl -name "*sl" -type f -print `

for X in $LIST
do
unset running
running=`fuser $X 2>>/dev/null`
if [ -n "${running}" ] ; then
print X is $X and has pids of $running
# I can even take it a down a level
for P in $running
do
ps -fp $P
done
echo "------------------------\n"
fi
done

I used this. With it, I discovered a dynamic shared object of perl was running with apache on 3 development websites. I stopped the 3 and whammo. No use anymore.

I've never used lsof before. If I did, I probably would replace the fuser command with it.

This script would be a performance hog. I would not not want to run it on a whim.

But I minimized the affect by just looking at files ending .sl. I could see from my first accident, (ahem), that these were the only ones in use.

Re: How to identify any processes running on a huge directory structure?

James R. Ferguson — Fri, 04 Mar 2011 14:16:05 GMT

Hi (again) Steve:

If you backup this directory with a utility that restores the lastaccesstime stamp of the files backed up, then consider using 'find' to see what has been accessed since some point in time.

For that matter, you still might glean some useful information if, for example, your complete backups were done weekly.

Regards!

...JRF...

Re: How to identify any processes running on a huge directory structure?

Steve Post — Fri, 04 Mar 2011 14:23:06 GMT

...why can't I do this???

cd /A
find perl -atime -1 -type f -print

if something shows up, I know it's in use?

Or
find perl -atime +600 -type f -print >old
find perl -type f -print > all
diff old all | more

Re: How to identify any processes running on a huge directory structure?

James R. Ferguson — Fri, 04 Mar 2011 14:35:26 GMT

Hi (again) Steve:

> # find perl -atime -1 -type f -print
> if something shows up, I know it's in use?

Well, you know that it *has been* *accessed* (read or executed) not necessarily that it is in-use.

Regards!

...JRF...

Re: How to identify any processes running on a huge directory structure?

Dennis Handly — Sat, 05 Mar 2011 15:24:15 GMT

>if something shows up, I know it's in use?

You can touch a reference file of an hour or so ago and then find files:
-neweram ref-file