Powerfull Find

[AZayed]

Dears,

Good day,

I have an issue with find command I would like to find files older than 2 days and move them to specific location. The issue is I want to build the exact same tree in the destination directory.

For example, I want to search /var/LOGS and move anything older than two days to /backup/var/LOGS. If there was a sub directories in the source I want to create it in the destination.

Regards,

[Jeeshan]

Salam Ahmed

try this command

#find /var/LOGS -type f -mtime +7 -exec mv -r {} /backup/var/LOGS \;

[Jeeshan]

mistake

#find /var/LOGS -type f -mtime +2 -exec mv -r {} /backup/var/LOGS \;

[James R. Ferguson]

Hi:

# cd /var
# find ./LOGS -depth -mtime +2 -print | cpio -pudlvm /backup/var

...will *copy* and replicate the directory/file structure in the destination directory. You will then need to remove what you don't want from the source tree in a separate pass.

Regards!

...JRF...

[Dennis Handly]

This solution does it in one pass but you'll need to write a customized script:

Starting from Ahsan's basic find:
find /var/LOGS -type f -mtime +2 -exec fancy_script {} +

Then create fancy_script:
#!/usr/bin/ksh
# Move files to /backup, keeping directory paths
for file in $*; do
mkdir -p /backup/$(dirname $file)
mv $file /backup/$file
done

[AZayed]

Thanks for reply,

Dear ahsan, mv doesn't have -r option it's only in cp.

Dears Dennis Handly and James R. Ferguson, I have this idea already, creating a script and calling it from find command. the script will use cp -r -p $1 $2 to create the specific tree. but I want to use mv command. This solution is very good but I prefer mv.

FANCY SCRIPT :

cp -r -p $1 $2
rm $1


FIND COMMAND :

find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT {} /backup/var/LOGS-DATE \;

Any suggestion.

Regards,

[OFC_EDM]

Thought I'd throw my hat in the ring with another idea.

Step 1. Copy the entire directory structure to the new location. Including the files you don't want.
"cp -r" or "cp -p -r" to preserve permisssions

Step 2. Nuke the files from the new structure which you don't want.

I'd test this out first by replacing rm with ls -l just to make certain....I don't have a system to test this on today and this is from memory.

Example:
Find files in current directory only older than June 3rd 2008

1. Create reference file
touch -t 200806030000 /tmp/june3.ref
Format of the numeric is YYYYMMDDHHMM
- use some logic in the script to calculate proper value to represent 2 days agos. I like this because you can control it down to the minute.

2 Run the command
find ./loc/A/* ! -type d ! -newer /tmp/june3.ref -exec rm {} \;

If you like it then stick this in a script to automate.

I'll try to find my script which has all the little things like calculating the date etc for the touch command. But it'll be a few days.

Cheers

[OFC_EDM]

I'll add one more option.

The previous example I put in goes into sub-directories.

However if at some point you only want to process the current directory stick in the -prune argument

find ./loc/A/* -prune ! -type d ! -newer /tmp/june3.ref -exec rm {} \;

Regards

[Steven E. Protter]

Shalom,

A word of caution.

All it takes is about 4 finds starting out at root to totally bring even a powerful system to its knees.

Use carefully, preferably one at a time.

Very interesting thread.

SEP

[Dennis Handly]

>I have this idea already, creating a script and calling it from find command. The script will use cp -r -p $1 $2 to create the specific tree. but I want to use mv command.

My script already does mv(1). What's wrong with it? It will only create the needed parts of the tree.

>find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT {} /backup/var/LOGS-DATE \;

If you are changing the path in your target, then you'll need to fiddle with my script a little:
find /var/LOGS -type f -mtime +2 -exec FANCY_SCRIPT /backup/var/LOGS-DATE {} +

FANCY_SCRIPT:
#!/usr/bin/ksh
TARGET=$1
shift
for file in $*; do
mkdir -p $TARGET/$(dirname $file)
mv $file $TARGET/$file
done

[James R. Ferguson]

Hi (again):

> Dears Dennis Handly and James R. Ferguson, I have this idea already, creating a script and calling it from find command.

Well, that's exactly what Dennis suggested originally to you! I presented an alternative, less direct answer to your original query (using 'find' and a pipe to 'cpio'). I think you have everything you need to make your decision.

Regards!

...JRF...

[AZayed]

Hi all, and thanks for your all ideas,

Dear Kevin, I would like to ask you about the "!" mark. if you put it before -newer option is it mean find older files not newer onec ?

I think I can solve this by moving the all directory to temporary directory and then find all the files newer than certain date as Kevin suggest and the returen those back with there directories to source directory. doing this I can save a lot of I/O because I'm dealing with more than 2,500,000 files.

-----------------

Again Kevin, thank for -prune option, I will write tomorowo script that have to go to every directory and count the number of files. So, I think this can help me with find.

-----------------

Dear JRF, I will read more about cpio, and feedback soon.

Thanks all.

[Dennis Handly]

>ask you about the "!" mark. if you put it before -newer option is it mean find older files not newer onec?

The opposite of newer is "NOT newer". Which is older or the same time.

>I think I can solve this by moving the all directory to temporary directory

What's wrong with my "mv" solution? (You can still use -newer if you want.)

[AZayed]

Hi folks,

I would like to ask about -mtime option.

In case you need to take all the files that were modified more than to days you need to use "-mtime +2". But this actually not finding all the files. Is this happened with you guys?

I have to put another find with -mtime 2 option to achieve my goal.

Any ideas ?

[Dennis Handly]

>I have to put another find with -mtime 2 option to achieve my goal.

What's your goal? -mtime +2 should be all files modified more than 2*24 hours ago.
Have you read find(1)?
http://docs.hp.com/en/B2355-60130/find.1.html

[AZayed]

Hi Dennis,

I want to know the difference between -mtime 1 & -mtime +1. In Sun +1 mean find all files that modified time is more than 24 hour.

Look at this:

touch -t 0810250000 file
touch -t 0810260000 file1
touch -t 0810270000 file2
touch -mt 0810270000 file3
touch -mt 0810260000 file4

#ls :

Oct 25 00:00 file
Oct 26 00:00 file1
Oct 26 00:00 file4
Oct 27 00:00 file2
Oct 27 00:00 file3

# find . -mtime +1
./file
# find . -mtime 1
./file1
./file4

----

I think "find . -mtime +1" suppose to display "find . -mtime 1" result.

Thanks

[Aneesh Mohan]

Hi Ahmed,


+n means more than n, -n means less than n, and n means exactly n.

here it is n * 24 hours.


Aneesh

[AZayed]

Hi,

So why the first command didn't find ./file1 , ./file4 files ?

Regards,

[Aneesh Mohan]

Hi Ahmad,

Try #find . -mtime +0 you will get it .

fyi:-
True if the file modification time subtracted
from the initialization time is n-1 to n
multiples of 24 h. The initialization time
shall be a time between the invocation of the
find utility and the first access by that
invocation of the find utility to any file
specified in its path operands.


Aneesh

[Dennis Handly]

>So why the first command didn't find ./file1, ./file4 files?

How can we tell if you don't tell us what time it is?
From find(1) there is a 24 hour fuzziness in -mtime n, perhaps there is for +n?

[Dennis Handly · Accepted solution]

find(1) works two ways depending if UNIX95 is exported. The find(1) describes UNIX95, which isn't the default.
time is now 27:1800
25:0000 2*24:1800 26:0000 1*24:1800 27:0000 0*24:1800
file X file1 X file2 NOW
Default: takes time / day and truncates
2 X 1 X 0
+0 X +0 X X
+1 X X X -1

UNIX95: takes time in seconds and compares N thru N-1 as =, - as < N-1, + as > N
file X file1 X file2 NOW
3 X 2 X 1
+0 X +0 X +0
+1 X +1 X X
+2 X X X -2

[Dennis Handly]

If you are happy with the answers you have gotten, please read the following about assigning points:
http://forums.itrc.hp.com/service/forums/helptips.do?#33

[AZayed]

Hi Dennis,

I didn't understand your last example, can you explain it a little.

Thanks

[Dennis Handly]

>can you explain it a little.

Which parts? How find(1) works with and without UNIX95?

Default:
25:0000 2*24:1800 26:0000 1*24:1800 27:0000 0*24:1800

This is a timeline, Oct 25, 0000; a time 2 days ago at 1800; then Oct 26, ... Oct 27, then "now at Oct 27, 1800.
Default: takes time / day and truncates

With UNIX95, it does this fuzzy compare that is documented in find(1).

[James R. Ferguson]

Hi (again):

> Dennis: With UNIX95, it does this fuzzy compare that is documented in find(1).

And just where in the HP-UX manpages did you find [ no pun intended ] that? I can't seem to locate a reference to either XPG4 or UNIX95 that points at this logic. The GNU Linux manpages are a bit more explicit in stating that "... When find figures out how many 24-hour periods ago the file was last accessed, any fractional part is ignored..."

Regards!

...JRF...