topic Re: script to move files in Operating System - Linux

script to move files

lawrenzo_1 — Fri, 14 Jul 2006 07:48:34 GMT

i am trying to write a function that checks the date of files in a directory and if the files are not equal to today then create the dir:

set -x
for i in `ls -l |awk '{print $6,$7}' |uniq`
do
if [ $i != $TODAY ] ; then
DIR=$($i |sed s/' '//)
mkdir $DIR
fi
done
+ ls -l
+ uniq
+ awk {print $6,$7}
+ [ 06 != 20 Jul ]
+ + sed s/ //
+ 06

how can i parse "06 Jun" into [ $i != $TODAY ]as one string?

Manyb Thanks

Re: script to move files

Jeff_Traigle — Fri, 14 Jul 2006 08:35:24 GMT

I don't see where you set TODAY. Setting it as such, should do the trick for you:

TODAY=$(date +"%d %b")

Re: script to move files

Enrico P. — Fri, 14 Jul 2006 09:15:05 GMT

Hi,
try with this:

ls -l |awk '{print $6,$7}' |uniq|while read i
do
echo $i
done

Enrico

Re: script to move files

Enrico P. — Fri, 14 Jul 2006 09:24:07 GMT

If you want exclude the directory from the ls command:

ls -l |grep -v ^d|awk '{print $6,$7}' |uniq|while read i
do
if [ $i != $TODAY ] ; then
DIR=$($i |sed s/' '//)
mkdir $DIR
fi
done

Enrico

Re: script to move files

Enrico P. — Fri, 14 Jul 2006 09:36:14 GMT

And you need to quote the variables in the:

if [ "$i" != "$TODAY" ]

Sorry
Enrico

Re: script to move files

Victor Fridyev — Fri, 14 Jul 2006 10:52:21 GMT

Hi,

If I understand you correctly, you need something like that:

#!/bin/sh
TODAY=$(date +%d%b)
/bin/ls -lt |grep -v ^d |while read FN; do
DATE=$(echo $FN|awk '{printf("%02d%s\n",$7,$6)}')
if [ "$DATE" != $TODAY ]; then
if [ ! -d $DATE ] ; then
mkdir $DATE
fi
fi
done
This works if all files in the current directory have been created on this year, otherwise you have to lok at $8, which is either year or HH:MM, but these are details.

HTH

Re: script to move files

Sandman! — Fri, 14 Jul 2006 12:10:03 GMT

Hi Lawrenzo,

Change the awk statement on the for-loop line so that it outputs a string w/o any embedded spaces. This also helps in compacting code since you don't need the sed statement anymore, which basically removes embedded spaces in $i, and make sure $TODAY contains the same date format as $i. Here's your code with the suggested modifications.

change -> awk '{print $6,$7}'
to -> awk '{print $6$7}'

remove -> DIR=$($i |sed s/' '//)

set -x
TODAY=$(date +"%b%d")
for i in `ls -l |awk '{print $6$7}' |uniq`
do
if [ $i != $TODAY ]; then
mkdir $i
fi
done

Re: script to move files

lawrenzo_1 — Mon, 17 Jul 2006 06:58:35 GMT

ok thats great to start my script off the main challenge I have here:

I want to list all files in the directory - if the directory doesn't exist create it, then move the relevant date files to the directory

if the directory does exist just move the files there.

TY

Re: script to move files

James R. Ferguson — Mon, 17 Jul 2006 07:17:40 GMT

Hi Lawrenzo:

I'd archive this way:

# touch -amt 07170000 /tmp/myref #...today
# mkdir /tmp/myarchive
# cd /tmp/myarchive
# find /path ! -newer /tmp/me|cpio -padmxv .

...This will find all files (and directories) that have been modified *before* today and copy then to your archive directory. A listing of the files copied is automatically generated. Modification and access times of the files are preserved.

Regards!

...JRF...

Re: script to move files

lawrenzo_1 — Mon, 17 Jul 2006 10:34:36 GMT

ok getting there now here is the function within the script

#!/bin/ksh

DIR=/usr/dump/JDE/data/out/ARCHIVE
LOGFILE=/sysadmin/logs/mvdata_JDE.log
REMOVE=/usr/dump/JDE/toberemoved
TODAY=$(date +"%d %b")

for a in `find $DIR -ctime 1 -type f`
do

month=`ls -l $a | awk -F' ' '{print $6$7}'`
file=`ls -l $a | awk -F' ' '{print $9}'`

if [ ! -d $month ] ; then

mkdir /usr/dump/JDE/data/out/ARCHIVE/$month
mv $file /usr/dump/JDE/data/out/ARCHIVE/$month

else
mv $file /usr/dump/JDE/data/out/ARCHIVE/$month
fi
done

this will create the directory and move the file.

The issue I have now is when the script runs the second day it still attempts to move all files that where moved the previous day.

how can i get find just to search the directory /usr/dump/JDE/data/out/ARCHIVE and not /usr/dump/JDE/data/out/ARCHIVE/day.mnth

thanks

Re: script to move files

Peter Nikitka — Mon, 17 Jul 2006 11:19:12 GMT

Hi,

the easiest thing would be to keep the monthly archive outside the find tree:

...
M_ARCHIVE=${DIR}-bymonth

[ -d $M_ARCHIVE ] || mkdir $M_ARCHIVE

for a in `find $DIR -ctime 1 -type f`
do
...
[ -d $M_ARCHIVE/$month ] || mkdir $M_ARCHIVE/$month
mv $file $M_ARCHIVE/$month
...
done

mfG Peter

Re: script to move files

James R. Ferguson — Mon, 17 Jul 2006 11:47:04 GMT

Hi (again) Lawrenzo:

You are using the 'ctime' or last change time for a file's inode. The 'ctime' changes whenever ownership, permissions are changed or whenever a file is renamed.

Using 'ctime' tracks metadata changes. If you truly want to find files based on *modification* of their data, you need to use 'mtime'.

The manpages for 'stat(2)' will clarify things for you.

BTW, I still suggest the 'cpio' method I posted above for your archiving needs.

Regards!

...JRF...

Re: script to move files

Sandman! — Mon, 17 Jul 2006 11:57:03 GMT

Modify the find command to exclude the other dir as follows:

find $DIR ! -path "/usr/dump/JDE/data/out/ARCHIVE/day.mnth/*" -ctime 1 -type f

~hope it helps

Re: script to move files

lawrenzo_1 — Tue, 18 Jul 2006 02:54:44 GMT

Thank you everyone - got there in the end!

James thanks for the suggestion however the business have request the archive directories this was.

I like your idea however it wouldn't work for what is required as there are over 3000 files daily that sometimes require a search etc etc.

Chris.