A shell problem

[Jayesh shah]

Hi,

I have a simple shell script of the form:

for var in list
do
--- write something into a file say TMP -----

--- read line by line and
------ for each line
do some actions.

The problem is the cotrol breaks from the inner loop even w/o completely reading the file .

It looks to be a problem with the shell buffer or something else?


The original script is below (MKDIR and CREATELINK are small shell script for creating dirs and links in clearcase environment)
I have highlighted the problem part in blue.
Can any one pinpoint the actaul cause for the problem?


==================================================================================================

set -xv
PRODUCT=$1
FLAVOR=$3
RELEASE=$2

BE_ROOT=/ha/be_sam
INFILE=$BE_ROOT/manifest.$PRODUCT.$FLAVOR
MAPPING=$HOME/MAPPING
TMP=$HOME/input$$
TARGET=$BE_ROOT/"$RELEASE"_$FLAVOR/

CREATEABELOG=$HOME/log/CREATEABELOG$$
CREATEABEERR=$HOME/log/CREATEABEERR$$
MKDIR=$HOME/bin/MKDIR
CREATELINK=$HOME/bin/CREATELINK

if [ ! -f $CREATEABELOG ]
then
touch $CREATEABELOG
else
rm $CREATEABELOG
touch $CREATEABELOG
fi

if [ ! -f $CREATEABEERR ]
then
touch $CREATEABEERR
else
rm $CREATEABEERR
touch $CREATEABEERR
fi

if [ ! -f $CREATEABELOG -o ! -f $CREATEABEERR ]
then
echo "$CREATEABELOG OR $CREATEABEERR COULD NOT BE CREATED"
exit 1
fi

if [ ! -f $MAPPING ]
then
echo "$MAPPING DOES NOT EXIST"
exit 1
fi

if [ ! -r $INFILE ]
then
echo "$INFILE DOES NOT EXIST OR IS NOT READABLE"
exit 1
fi


for SRC in `grep "augmentation level" $INFILE |awk '{print $1}'`
do
sed -n '/'$SRC' augmentation level/,/augmentation level/p' $INFILE | sed '/augmentation level/d' |sed '/rule/d' >$TMP

ROOTSRCDIR=`grep $SRC $MAPPING | cut -f2`

echo "=============================="

cat $TMP | while read LINE
do
FILE=`echo $LINE|awk '{ print $3 }'`
DIR=`dirname $FILE`
echo "LINE ....$LINE"
if [ ! -d $TARGET/$DIR ]
then
$MKDIR $TARGET/$DIR $CREATEABELOG $CREATEABEERR
fi

if [ -f $ROOTSRCDIR/$FILE ]
then
RESULT=`cksum $ROOTSRCDIR/$FILE | awk '{ print $1 $2 }'`
LINE12=`echo $LINE | awk '{ print $1 $2 }'`
if [ "$RESULT" = "$LINE12" ]
then
$CREATELINK $ROOTSRCDIR/$FILE $TARGET/$FILE $CREATEABELOG $CREATEABEERR
else
echo "$ROOTSRCDIR/$FILE has checksum problems" >> $CREATEABEERR
fi
else
echo "$ROOTSRCDIR/$FILE does not exist" >> $CREATEABEERR
fi


done




done
=======================================================================================

[Jayesh shah]

The problem part is
cat $TMP | while read LINE

....

Thanks,
Jayesh

[harry d brown jr]

You are probably right about the buffer. run this again, but this time do a "set -x" so that you can watch it. if it is a buffer issue, it'll never do the read LINE. This also look like a great opportunity to learn perl.


live free or die
harry

[Jayesh shah]

I had tried set -x.
It does read LINE,
but i do not know it breaks all of a sudden to the outer loop.
I do not understand the reason.Well,Is it easy to write the same in perl?

[James R. Ferguson]

Hi Jayesh:

I think the essential problem is that your outer 'do' loop is writing to the TMP file, while the inner loop is attempting to read the same file. I suggest that you rearrange your code into two indepenedent passes -- the first to write to the file; the second to read the updated information.

Regards!

...JRF...

[harry d brown jr]

This is close, but because I donw't have your system to test on, it's not exact.

live free or die
harry

#!/usr/bin/perl???? you need to put the path to perl here

$PRODUCT=$1;
$FLAVOR=$3;
$RELEASE=$2;

$BE_ROOT=/ha/be_sam;
$INFILE=$BE_ROOT/manifest.$PRODUCT.$FLAVOR;
$MAPPING=$HOME/MAPPING;

$TARGET=$BE_ROOT/"$RELEASE"_$"FLAVOR/";

$CREATEABELOG=$HOME/log/CREATEABELOG$$;
$CREATEABEERR=$HOME/log/CREATEABEERR$$;
$MKDIR=$HOME/bin/MKDIR;
$CREATELINK=$HOME/bin/CREATELINK;

eval `cat /dev/null >$CREATEABELOG` or die "Could not null $CREATEABELOG";
open (ABELOG,">>$CREATEABELOG") || die "Could not open $CREATEABELOG for appending";
eval `cat /dev/null >$CREATEABEERR` or die "Could not null $CREATEABEERR";
open (ABEERR,">>$CREATEABEERR") || die "Could not open $CREATEABEERR for appending";

open (FILEHANDLE, $MAPPING ) || die "MAPPING DOES NOT EXIST";
open (FILEHANDLE, $INFILE ) || die "INFILE DOES NOT EXIST OR IS NOT READABLE";
open (SRClist, `grep "augmentation level" $INFILE |awk '{print $1}'|` ) || die "No data to process";

while ($src=) {
chop($SRC);

open (TMP, `sed -n '/'$SRC' augmentation level/,/augmentation level/p' $INFILE | sed '/augmentation level/d' |sed '/rule/d'|` ) || die "No data to process";

$ROOTSRCDIR=`grep $SRC $MAPPING | cut -f2`;

print "==============================\n";

while ($LINE = ) {
chop($LINE);

$FILE=`echo $LINE|awk '{ print $3 }'`;
$DIR=`dirname $FILE`;
print "LINE ....$LINE\n";

if (! -d "$TARGET/$DIR" ) {
`$MKDIR $TARGET/$DIR $CREATEABELOG $CREATEABEERR`;
}

if ( -f "$ROOTSRCDIR/$FILE" ) {
$RESULT=`cksum $ROOTSRCDIR/$FILE | awk '{ print $1 $2 }'`;
$LINE12=`echo $LINE | awk '{ print $1 $2 }'`;
if ( $RESULT eq $LINE12 ) {
eval `$CREATELINK $ROOTSRCDIR/$FILE $TARGET/$FILE $CREATEABELOG $CREATEABEERR` or die "Could not create link";
} else {
print ABEERR "$ROOTSRCDIR/$FILE has checksum problems\n";
}
} else {
print ABEERR "$ROOTSRCDIR/$FILE does not exist\n";
}
}
}

[Jayesh shah]

I did rearrange the code into two indepenedent passes -- the first to write to the file; the second to read the updated information.

But the problem remains.

[Jayesh shah]

The perl script gives a lot of syntax like bare? something.

[harry d brown jr]

I'll connect into one of my servers and do a little testing. Hopefully I'll have an answer today, or at least by monday morn.

live free or die
harry

[harry d brown jr · Accepted solution]

Here, I modified the syntax - damn typos....

its in the attachment


live free or die
harry