mass file move / rename -

[rmueller58]

We have a set of documents that are loaded into a Job applicant webserver.

We have a consistent Naming convention

Currently we have files that are named based on

__.ext

for each there are several hundred items, for each there maybe from 1 to 10 ..

We are upgrading to a new revision of the web application but need to create a script that will copy the file from one name to the "new name"

As we can select which directory we can place the file, we no longer require the
The vendor has added an iteration #, we are not using this so the value will always be '0'.

The New File Name string will strip off the , and place an "_O" zero between and

so the new file name would be as follows:

.ext

I need to take the existing files and copy them from our existing folder and place them with the new file name in the new subdirectory.

Any ideas on how best to handle this.. There are 13000 documents.

[Hasan Atasoy]

hi


for i in `ls -1 DISTID*`
do
filename=`echo $i | tr -d DISTID`
echo $i $filename
done

if echo command ok then replace echo by mv .


Hasan

[Matti_Kurkela]

Since this is a one-time job, there is probably no need to find a perfectly efficient solution.

The "find" command will produce a list of filenames, one per line, without wasting time in sorting them.

Pipe that to a "while read FILENAME; do ... done" style loop.

Within the loop, you'll get the current filename (including the path) in $FILENAME.

There are various ways to split it into components; the "cut" command might be the easiest.

"basename $FILENAME | cut -d _ -f 1" outputs if you'll need it for anything, change the value of the option -f to 2 to get the and to 3 to get the .ext part.

"dirname $FILENAME" will output the directory path without the filename.

Capture the output into variables by using the "VARIABLE=$( commandline )" syntax.

Now you should have a series of variables you can use to assemble the necessary mv command within the loop. Consider using the -i option of cp so that you won't accidentally overwrite any files.

Test your script by prefixing the cp command line with "echo" so that it only displays what it would do.

When you're made sure it works correctly, remove the "echo" prefix, and start the script for real.

MK

[Hein van den Heuvel]

Hmm, just yesterday a similar question was asked and its replies may help you.

http://forums11.itrc.hp.com/service/forums/questionanswer.do?threadId=1308418

For example (untested)

$ cd
$ mv /*.* .
$ perl -e 'for (@ARGV) { $old=$_; if( /.*?_([^_]+)(.*)/) { $new=$1 . q(_0_) . $2; print qq(rename $old, $new\n)} }' *.tmp

Remove the print( \n) to make it real:

Hein.

[rmueller58]

Hein,

I am pretty perl and regex generally illiterate. Forgive me for responding slowly other issues came up.

I have a directory with files that have a naming convention as follows:

mpsplus_1234_1.ext

the new value should be
1234_0_1.ext

Basically in need to string the mpsplus_ from the front and insert a "0_ after the 1234. The zero is non-changing value but required by the database to index the document correctly.

for each of 13000 files ..

the consistent thing is the current mpsplus_
everything all things being equal we want to retain the 1234_"0_"1.ext

Based on your script Perl will issue an edit "-e" for values in @ARGV
Does ARGV assume files in the current working directory? or do I need to be more explicit.. I have copied some files into a test directory I will attempt with ..

DO I remove the entire
"print qq(rename $old, $new\n"

To see if it works? I got no output when I ran it as shown. is there a way to echo out a file per file change?


perl -e 'for (@ARGV) { $old=$_; if( /.*?_([^_]+)(.*)/) { $new=$1 . q(_0_) . $2; print qq(rename $old, $new\n)} }' *.tmp

[Mel Burslan · Accepted solution]

I know this is not the most elaborate way of doing this or most efficient way, or most anything for that matter, but again since this is a one time thing and 13000 is a relatively small number of files despite how huge it seems to you, the following solution should work.

for file in `ls -1 mpsplus_*`
do
field2=`echo $file | cut -d"_" -f2`
field3=`echo $file | cut -d"_" -f3|cut -d"." -f1`
ext=`echo $file|cut -d"." -f2`
newfile=${filed2}_0_${field3}.${ext}

echo ${file} ${newfile}
#mv ${file} ${newfile}

done

if the output of the echo is looking fine, just comment move the # sign in front of echo line and remove it from mv line.

Hope this helps

[rmueller58]

Mel,

Get the following error:
-bash: /bin/ls: Argument list too long


Any thoughts?

[rmueller58]

Mel,

I resolved the ls issue by going the "find" command
for file in `find . -name "mpsplus*" -print |awk '{print substr($1,3,30)}'`
do
field2=`echo $file | cut -d"_" -f2`; field3=`echo $file | cut -d"_" -f3|cut -d"." -f1`
ext=`echo $file|cut -d"." -f2`; newfile=${field2}_0_${field3}.${ext}
mv ${file} ${newfile}
done

Thanks all for the assistance!! Points assigned

[Mel Burslan]

Then you need to handle the pipe in good old manual way as follows:
(again, there should be a more elaborate way of doing this but for all intents and purposes, this should work)

ls -1 mpsplus_* >/tmp/filelist.txt
echo "QQQ" >> /tmp/filelist.txt
counter=0 # for visual entertainment purposes
file=dummy # initial value

while [ "$file" != "QQQ" ]
do
file=`head -1 /tmp/filelist.txt`

field2=`echo $file | cut -d"_" -f2`
field3=`echo $file | cut -d"_" -f3|cut -d"." -f1`
ext=`echo $file|cut -d"." -f2`
newfile=${filed2}_0_${field3}.${ext}

echo ${file} ${newfile}
#mv ${file} ${newfile}
(( counter=${counter}+1 )); echo ${counter}

cat /tmp/filelist.txt | sed -e "1,1d" > /tmp/filelist.tmp
mv /tmp/filelist.tmp /tmp/filelist.txt

done

on the last line, where it will read the filename QQQ, the mv command will give you a file not found error but it is not a problem, just cosmetic detail.

what this construct does is reads the top line, processes filename then deletas that line from the file and repeats until the file gets exhausted. The display of counter value will show you the progress and you can figure out how fast or slow it is going.

good luck

[rmueller58]

Seems I've had problems with using ls in the past with a similar script.

I resolved it with the find command, like you I am sure there are dozens of more elegant ways of doing things but it is doing the job. Thanks again all of you..

rexm