topic Re: Powerfull Find in Operating System - HP-UX

Powerfull Find

AZayed — Sun, 21 Sep 2008 11:36:51 GMT

Dears,

Good day,

I have an issue with find command I would like to find files older than 2 days and move them to specific location. The issue is I want to build the exact same tree in the destination directory.

For example, I want to search /var/LOGS and move anything older than two days to /backup/var/LOGS. If there was a sub directories in the source I want to create it in the destination.

Regards,

Re: Powerfull Find

Jeeshan — Sun, 21 Sep 2008 11:52:16 GMT

Salam Ahmed

try this command

#find /var/LOGS -type f -mtime +7 -exec mv -r {} /backup/var/LOGS \;

Re: Powerfull Find

Jeeshan — Sun, 21 Sep 2008 11:54:00 GMT

mistake

#find /var/LOGS -type f -mtime +2 -exec mv -r {} /backup/var/LOGS \;

Re: Powerfull Find

James R. Ferguson — Sun, 21 Sep 2008 12:21:36 GMT

Hi:

# cd /var
# find ./LOGS -depth -mtime +2 -print | cpio -pudlvm /backup/var

...will *copy* and replicate the directory/file structure in the destination directory. You will then need to remove what you don't want from the source tree in a separate pass.

Regards!

...JRF...

Re: Powerfull Find

Dennis Handly — Sun, 21 Sep 2008 18:53:07 GMT

This solution does it in one pass but you'll need to write a customized script:

Starting from Ahsan's basic find:
find /var/LOGS -type f -mtime +2 -exec fancy_script {} +

Then create fancy_script:
#!/usr/bin/ksh
# Move files to /backup, keeping directory paths
for file in $*; do
mkdir -p /backup/$(dirname $file)
mv $file /backup/$file
done

Re: Powerfull Find

AZayed — Mon, 22 Sep 2008 06:03:40 GMT

Thanks for reply,

Dear ahsan, mv doesn't have -r option it's only in cp.

Dears Dennis Handly and James R. Ferguson, I have this idea already, creating a script and calling it from find command. the script will use cp -r -p $1 $2 to create the specific tree. but I want to use mv command. This solution is very good but I prefer mv.

FANCY SCRIPT :

cp -r -p $1 $2
rm $1

FIND COMMAND :

find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT {} /backup/var/LOGS-DATE \;

Any suggestion.

Regards,

Re: Powerfull Find

OFC_EDM — Mon, 22 Sep 2008 06:32:21 GMT

Thought I'd throw my hat in the ring with another idea.

Step 1. Copy the entire directory structure to the new location. Including the files you don't want.
"cp -r" or "cp -p -r" to preserve permisssions

Step 2. Nuke the files from the new structure which you don't want.

I'd test this out first by replacing rm with ls -l just to make certain....I don't have a system to test this on today and this is from memory.

Example:
Find files in current directory only older than June 3rd 2008

1. Create reference file
touch -t 200806030000 /tmp/june3.ref
Format of the numeric is YYYYMMDDHHMM
- use some logic in the script to calculate proper value to represent 2 days agos. I like this because you can control it down to the minute.

2 Run the command
find ./loc/A/* ! -type d ! -newer /tmp/june3.ref -exec rm {} \;

If you like it then stick this in a script to automate.

I'll try to find my script which has all the little things like calculating the date etc for the touch command. But it'll be a few days.

Cheers

Re: Powerfull Find

OFC_EDM — Mon, 22 Sep 2008 06:47:14 GMT

I'll add one more option.

The previous example I put in goes into sub-directories.

However if at some point you only want to process the current directory stick in the -prune argument

find ./loc/A/* -prune ! -type d ! -newer /tmp/june3.ref -exec rm {} \;

Regards

Re: Powerfull Find

Steven E. Protter — Mon, 22 Sep 2008 07:14:50 GMT

Shalom,

A word of caution.

All it takes is about 4 finds starting out at root to totally bring even a powerful system to its knees.

Use carefully, preferably one at a time.

Very interesting thread.

SEP

Re: Powerfull Find

Dennis Handly — Mon, 22 Sep 2008 08:19:37 GMT

>I have this idea already, creating a script and calling it from find command. The script will use cp -r -p $1 $2 to create the specific tree. but I want to use mv command.

My script already does mv(1). What's wrong with it? It will only create the needed parts of the tree.

>find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT {} /backup/var/LOGS-DATE \;

If you are changing the path in your target, then you'll need to fiddle with my script a little:
find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT /backup/var/LOGS-DATE {} +

FANCY_SCRIPT:
#!/usr/bin/ksh
TARGET=$1
shift
for file in $*; do
mkdir -p $TARGET/$(dirname $file)
mv $file $TARGET/$file
done

Re: Powerfull Find

James R. Ferguson — Mon, 22 Sep 2008 10:41:52 GMT

Hi (again):

> Dears Dennis Handly and James R. Ferguson, I have this idea already, creating a script and calling it from find command.

Well, that's exactly what Dennis suggested originally to you! I presented an alternative, less direct answer to your original query (using 'find' and a pipe to 'cpio'). I think you have everything you need to make your decision.

Regards!

...JRF...

Re: Powerfull Find

AZayed — Mon, 22 Sep 2008 17:47:29 GMT

Hi all, and thanks for your all ideas,

Dear Kevin, I would like to ask you about the "!" mark. if you put it before -newer option is it mean find older files not newer onec ?

I think I can solve this by moving the all directory to temporary directory and then find all the files newer than certain date as Kevin suggest and the returen those back with there directories to source directory. doing this I can save a lot of I/O because I'm dealing with more than 2,500,000 files.

-----------------

Again Kevin, thank for -prune option, I will write tomorowo script that have to go to every directory and count the number of files. So, I think this can help me with find.

-----------------

Dear JRF, I will read more about cpio, and feedback soon.

Thanks all.

Re: Powerfull Find

Dennis Handly — Mon, 22 Sep 2008 17:54:57 GMT

>ask you about the "!" mark. if you put it before -newer option is it mean find older files not newer onec?

The opposite of newer is "NOT newer". Which is older or the same time.

>I think I can solve this by moving the all directory to temporary directory

What's wrong with my "mv" solution? (You can still use -newer if you want.)

Re: Powerfull Find

AZayed — Mon, 27 Oct 2008 05:31:18 GMT

Hi folks,

I would like to ask about -mtime option.

In case you need to take all the files that were modified more than to days you need to use "-mtime +2". But this actually not finding all the files. Is this happened with you guys?

I have to put another find with -mtime 2 option to achieve my goal.

Any ideas ?

Re: Powerfull Find

Dennis Handly — Mon, 27 Oct 2008 06:05:26 GMT

>I have to put another find with -mtime 2 option to achieve my goal.

What's your goal? -mtime +2 should be all files modified more than 2*24 hours ago.
Have you read find(1)?
http://docs.hp.com/en/B2355-60130/find.1.html

Re: Powerfull Find

AZayed — Mon, 27 Oct 2008 08:40:36 GMT

Hi Dennis,

I want to know the difference between -mtime 1 & -mtime +1. In Sun +1 mean find all files that modified time is more than 24 hour.

Look at this:

touch -t 0810250000 file
touch -t 0810260000 file1
touch -t 0810270000 file2
touch -mt 0810270000 file3
touch -mt 0810260000 file4

#ls :

Oct 25 00:00 file
Oct 26 00:00 file1
Oct 26 00:00 file4
Oct 27 00:00 file2
Oct 27 00:00 file3

# find . -mtime +1
./file
# find . -mtime 1
./file1
./file4

----

I think "find . -mtime +1" suppose to display "find . -mtime 1" result.

Thanks

Re: Powerfull Find

Aneesh Mohan — Mon, 27 Oct 2008 08:54:33 GMT

Hi Ahmed,

+n means more than n, -n means less than n, and n means exactly n.

here it is n * 24 hours.

Aneesh

Re: Powerfull Find

AZayed — Mon, 27 Oct 2008 09:28:31 GMT

Hi,

So why the first command didn't find ./file1 , ./file4 files ?

Regards,

Re: Powerfull Find

Aneesh Mohan — Mon, 27 Oct 2008 10:25:33 GMT

Hi Ahmad,

Try #find . -mtime +0 you will get it .

fyi:-
True if the file modification time subtracted
from the initialization time is n-1 to n
multiples of 24 h. The initialization time
shall be a time between the invocation of the
find utility and the first access by that
invocation of the find utility to any file
specified in its path operands.

Aneesh

Re: Powerfull Find

Dennis Handly — Mon, 27 Oct 2008 11:56:47 GMT

>So why the first command didn't find ./file1, ./file4 files?

How can we tell if you don't tell us what time it is?
From find(1) there is a 24 hour fuzziness in -mtime n, perhaps there is for +n?