Perl File manipulation

[Srikanth Arunachalam]

Hi,

I already have a shell script with logic of
FEED_SEARCH_FILES=$RECEPTION/CPR_??????????_G${GDG}V??.DAT

for f in `ls ${FEED_SEARCH_FILES}`
do
FILENAME=`basename ${f}`
DATE=`echo ${FILENAME} | cut -b"5-14"`
if [ "EMPTY_FILE" != "${DATE}" ]
then
FILE_LIST="${FILE_LIST} ${f}"
fi
FEEDFILECOUNT=`expr ${FEEDFILECOUNT} + 1`
done

if [ ${FEEDFILECOUNT} -ne ${EXPECTED_FEED_FILE_COUNT} ]
then
Abort "The number of feed files in the product roles feed is ${FEEDFILECOUNT} when ${EXPECTED_FEED_FILE_COUNT} were expected" ${LINENO}
fi

I would like to have the equivalent in perl script please?

Any earlier response will be much apprecialted.

[James R. Ferguson · Accepted solution]

Hi:

> I would like to have the equivalent in perl script please?

First, why? Is your script doing what you want it to? I seriously doubt it.

With all due respect, you need to learn to write shell scripts first.

Avoid the deprecated backticks and use the Posix syntax:

# FILENAME=$(basename ${f})
# DATE=$(echo ${FILENAME}|awk '{print $6,$7}')

Notice that your 'cut' would not have snipped the date (month & day) from the 'ls' output since it doesn't begin at byte five. Since 'cut's delimiter is one character you can't use it to snip fields that are separated by multiple delimiters in succession. Hence, the 'awk'.

Using 'expr' to perform arithmetic is very slow invokes an extra process where none is needed. Instead of:

# FEEDFILECOUNT=`expr ${FEEDFILECOUNT} + 1`

Use:

# FEEDFILECOUNT=$((FEEDFILECOUNT+1))

This is described in the manpages for 'sh-posix'.

Regards!

...JRF...

[Dennis Handly]

>JRF: Use: FEEDFILECOUNT=$((FEEDFILECOUNT+1))

You can simplify it farther:
(( FEEDFILECOUNT = FEEDFILECOUNT+1 ))
Or:
(( FEEDFILECOUNT += 1 ))

>DATE=$(echo ${FILENAME} | awk '{print $6,$7}')
>Notice that your cut(1) would not have snipped the date (month & day) from the 'ls' output

This is ls(1) not ll(1) output and is a just snipping the chars out of the filename, that "??????????" string.
In this case awk's substr($1, 5, 10) would work. Or use expr(1) substr:
DATE=$(expr substr ${FILENAME} 5 10)

[Dennis Handly]

Oops: Or use expr(1) substr:
DATE=$(expr substr ${FILENAME} 5 10)

[James R. Ferguson]

Hi (again):

OK, my apologies for not reading better. Dennis is correct, I read 'ls -l' instead of a simple 'ls' of only filenames.

That said, again, why do you want to redraft your script in Perl?

If you insist on doing this you could use 'readdir()' to read the directory in question or you could do a piped open of 'll'. The pattern matching would then select or reject filenames meeting your criteria.

Regards!

...JRF...

[Srikanth Arunachalam]

Hi James,

Good thought. Actually it was not scripted by me. Its been there for ages and hence do not make use of posix. I believe perl would be lot quicker, correct me if I am wrong.

Thanks,
Srikanth

[James R. Ferguson]

Hi:

You would need to benchmark but using Perl might be faster for large numbers of files. Your shell code might look something like this in Perl:

# cat ./mysearch
#!/usr/bin/perl
use strict;
use warnings;
my $match = qr(CPR_(.{15})_G$ENV{GDG}V..\.DAT);
my $count;
my @myfiles;
open( my $fh, '-|', 'ls' ) or die "Can't fork: $!\n";
while (<$fh>) {
next unless $_ =~ $match;
push @myfiles, $_ if $1 !~ /EMPTY_FILE/;
$count++;
}
print $_ for (@myfiles);
if ($count != $ENV{COUNT}) {
die "count = $count not equal expected ($ENV{COUNT})\n";
}
1;

# export COUNT=101 #...whatever is expected...
# export GDG=somestring #...whatever you need to match...

...and run as:

# ./mysearch

For simplicity and as an exercise for you, I left the addition of the 'RECEPTION" environmental variable.

Again, my apologies for not reading your original post most correctly before replying.

Regards!

...JRF...

[Srikanth Arunachalam]

Hi James,

You are worth more than the 10 points, I would not be able to give anything beyond that :-). There cannot be any score matching your humbleness.

Please dont worry about this anymore.

Thanks,
Srikanth A

[H.Merijn Brand (procura]

no need to keep count :)

my $count = @files;

will return the number of elements in @files (array used in scalar context);

If you either have no subfolders or if you actually want to descent into the folder structure, readdir (or glob ()) is most likely not the path to follow

--8<---
use strict;
use warnings;
use File::Find

my @files;

my $match = qr(CPR_(.{15})_G$ENV{GDG}V..\.DAT);
find (sub {
m/PR_(.{15})_G$ENV{GDG}V..\.DAT/ &&
$1 ne "EMPTY_FILE" and
push @files, $File::Find::name;
}, ".");

print join "\n", @files, "";
scalar @files == $ENV{COUNT} or
die "count = $count not equal expected ($ENV{COUNT})\n";
-->8---

Enjoy, Have FUN! H.Merijn