String handling in shell scripts

[Dennis Handly]

You can use mixed solution of shell read and awk to do what you're doing. But use awk to split the fixed length fields:
while read line; do
# Your solution, with fixes
set $line
if [ $# -eq 2 ]; then
a=$1
b=$2
else
a=$(echo "$line" | cut -c 1-6)
b=$(echo "$line" | cut -c 7-12)
fi
buffer=$(awk -v a="$a" -v b="$b" '{ if( $2 == a && $3 == b)
print $2, $3, $4, $6, $7, $8, $9, $10, $11}' ${file2})
echo $buffer >> ${ifn}

# awk solution
awk -v line="$line" '
BEGIN {
a = substr(line, 1, 6)
gsub(" ", "", a) # remove spaces
b = substr(line, 7, 6)
gsub(" ", "", b)
}
{
if ($2 == a && $3 == b)
print $2, $3, $4, $6, $7, $8, $9, $10, $11
}' ${file2} >> ${ifn}
done < $file1

>The problem I'm dealing with is that the shell seems to drop leading blanks.

What do you want to do with those leading blanks? You won't find them in your second file.

[J Ruud]

My mixed solution of shell and awk was working fine except for one case, when the first field had leading spaces and the second field didn't. Then, the number of args would be one and I'd have to use the cut to parse the fields. The problem with the shell dropping the leading white space was that the "cut 1-6" would then grab data from the second field. I wasn't worried about the leading space in awk. It was about getting the fields parsed correctly.

[J Ruud]

Dennis,
Regarding your solution, it gives me an idea of another approach that I might take using awk. But, I don't think it would have worked as you have it coded. You see, with the space being dropped, the awk script would only get a string of 11 bytes. So the substr would have the same problem as my cut.

[James R. Ferguson]

Hi:

Instead of reading a line into a single variable, I would do:

...
while read A B X
...

Now, either you have two fields (or more, in which case the third..n-th are in 'X') or you have field 'A' with an empty 'B'.

If you have one field ('B' is empty), you could reject the record read if the size of 'A' isn't exactly 12-characters (your requirement).

For instance, you could issue an error message and continue the 'read' loop by adding:

[ "${#A}" -ne 12 ] && { echo "bad_size"; continue; }

If you are going to change the Inter-Field-Seperator to prevent field splitting and preserve both leading and trailing spaces, *at least* localize its action:

#!/usr/bin/sh
OLDIFS=${IFS}
IFS=''
while read A
do
echo "[${A}]"
echo "...and my size was: " ${#A}
done
IFS=${OLDIFS}

...

Regards!

...JRF...

[J Ruud]

Thanks for your input. The file will always have two fields, unless I change it. In case you're wondering, the format is such because of the number of different programs and files with which it is used. I'm not that experienced with programming in shell. Mainly, I've used it to call other programs and I learn as much as I need to get the job done. This whole exercise started when I decided to provide added functionality by modifying this script to take in a file of arguments, instead of the original two. So, first, I had to figure out how to read the file and pass the args to the awk script. Your suggestion shows me another way that shell can be used to perform file processing tasks. Also, I like the idea of localizing the IFS influence. And, although I don't have to worry about it having a negative impact on the processing of this script, I'll definitely implement your idea. Thanks to everyone for your responses. It's been a positive learning experience.

[Dennis Handly]

>with the space being dropped, the awk script would only get a string of 11 bytes. So the substr would have the same problem as my cut.

Yes, you need that IFS solution too. But you need to later remove those spaces.