script challenge - file moving

[Jim Mickens]

I need to create a script that will move files based on information in the file name. I have a temporary directory in which a bunch of .zip files reside. Each zip file has a job number in the file name, and I need to be able to move them to the appropriate production directory based on the job number information in the file name. For example:
# ll /prod/temp/ascribe_archives
total 17124
-rw-rw-rw- 1 jmickens users 351465 Jun 26 14:31 20030115_j341281_Morgan_Stanley_Retail_RDD_Q1_2003.zip
-rw-rw-rw- 1 jmickens users 137011 Jun 26 14:31 20030120_j341271_Morgan_Stanley_Heavy_Up_RDD_Q1_2003.zip

The first file needs to be moved to /prod/job1/j341281, and the second needs to be moved to /prod/job1/j341271.

The job number will always appear between the first two underscores, but may not always be the same length (some might have 5 digits, others 6). I also need to leave the file where it is if the production directory that it goes with doesn't already exist. The reason I need to automate this is that these are files that are downloaded from a supplier ftp site every week.

I'm not a high-level script programmer. Any suggestions would be greatly appreciated.

[Robert-Jan Goossens]

Hi,

cd /prod/temp/ascribe_archives
ls | grep j341281 | while read LINE
do
mv $LINE /prod/job1/j341281/
done


cd /prod/temp/ascribe_archives
ls | grep j341271 | while read LINE
do
mv $LINE /prod/job1/j341271/
done

Robert-Jan.

[Ian Dennison_1 · Accepted solution]

for i in $(ls /prod/temp/ascribe_archives)
do
echo ARCH_DIR=`echo $i |sed 's/_/ /g' |awk '{print $2}'`

if [[ ! -d /prod/job1/$ARCH_DIR ]]
then
# Directory does not exist
mkdir /prod/job1/$ARCH_DIR
if (( $? < 1 ))
then
print "error"
fi
fi

cp $i /prod/job1/$ARCH_DIR
if [[ -f /prod/job1/$ARCH_DIR/$i ]]
then
rm $i
else
echo "could not move file $i"
fi

done

This script ensures that the file is not deleted until it is a valid entry in the sub-directory.

Share and Enjoy! Ian

[Ian Dennison_1]

Sorry,

if (( $? < 1 ))

should read

if (( $? > 0 ))

Share and Enjoy! Ian

[Jean-Louis Phelix]

Hi,

#!/usr/bin/sh

cd /prod/temp/ascribe_archives
for FILE in $(ls 2>&-)
do
JOB=$(echo $FILE | sed 's;^[^_]*_\([^_]*\)_.*$;\1;')
[ -d "/prod/job1/$JOB ] || continue
mv $FILE /prod/job1/$JOB
done

Regards.

[Massimo Bianchi]

I would do something like this:


LIST=$( find /prod/temp/ascribe_archives -type f -print)

for i in $LIST
do
# this is the dest directory
DEST_DIR=$( echo $i| awk -F_ '{print $2}' )
FULL_DIR=/prod/job1/$DEST_DIR
if [-d $FULL_DIR]
then
mv $i $FULL_DIR
else
echo $FULL_DIR not found, skipping file $i
fi

done


HTH,
Massimo

[James R. Ferguson]

Hi Jim:

This will get you started:

#!/usr/bin/sh
cd /tmp
ls -l 2003*|awk '{print $NF}'|while read F
do
FF=`echo ${F}|awk -F"_" '{print $2}'`
echo cp ${F} /prod/job/${FF}
done
exit 0

I have assumed that your source files reside in the '/tmp' directory in the above example. I have also assumed that they are named starting with the string "2003". Drop the 'echo' in front of the 'cp' and files will actually be copied.

Regards!

...JRF...

[Massimo Bianchi]

Hi,
a little futher check to add, at everyone's reply (mine inclusive):

let's suppone that $i is source

...
cp $i $DEST_DIR/$i
diff $i $DEST_DIR 1>/dev/null 2>&1 && rm $i
...


so you are sure that the file copied was not being edited :)


cheers,
Massimo

[Massimo Bianchi]

Hi,
a little futher check to add, at everyone's reply (mine inclusive):

let's suppone that $i is source

...
cp $i $DEST_DIR/$i
diff $i $DEST_DIR/$i 1>/dev/null 2>&1 && rm $i
...


so you are sure that the file copied was not being edited :)


cheers,
Massimo

[Jim Mickens]

So many suggestions, and so quickly! Now I need time to absorb and test. Thanks everyone. I'll let you know how it turns out.

[Shannon Petry]

I guess we have to "assume" that the second field is always the job number.

This may be lengthy, but I like to see what's going on and keep it simple for the next person.

#!/bin/sh
ARCH_DIR=/prod/temp/ascribe_archives
MV_BASE=/prod/job1
LOG=/tmp/file_mover.log
if [ -f $LOG ] ; then
rm -f $LOG
fi
touch $LOG
cd $ARCH
for FILE in `ls` ; do
NEWDIR=`echo $FILE|awk -F_ '{print $2}'`
if [ -d ${MV_BASE}/$NEWDIR ] ; then
echo "Moving ${ARCH_DIR}/${FILE} to ${MV_BASE}/${NEWDIR}" >>$LOG
mv $FILE ${MV_BASE}/$NEWDIR
else
echo "No such Directory ${MV_BASE}/${NEWDIR}" >>$LOG
fi
done


Regards,
Shannon

[Jim Mickens]

Got it working. Thanks to everyone for thier suggestions. I couldn't have done it without you! I wound up using a modified version of the script Ian posted. I have attached it for anyone interested. Works like a charm.