Read variables from file

[Amar_Joshi]

Hi,

I have a file with thousand rows (not columns) in it. I want to read each line (one by one) and then use it as command syntax in another script. How can I do that?

Currently I have this but it doesn't work...

while [ $MYCOUNT -lt $CAIDCOUNT ]
do
CAID=`awk -F '/n' '{print $1}' varlist.txt`
echo "$MYCOUNT $CAID" > out.txt
let MYCOUNT=$MYCOUNT+1
done

Thanks in advance

[Tim Nelson · Accepted solution]

Awk is reading the entire file line by line.
To do it this way you would have to keep incrementing NR until done.


If there are no spaces then a while read X loop would work.

Can you supply an example of your input file.

[Sandman!]

Not quite clear what you mean from the code snippet posted here. Could you provide a sample of the input file and also give a clearer and more indepth explanation of what you are trying to accomplish.

~thanks

[Dennis Handly]

What are you really trying to do?
If you intended to have your loop above read CAIDCOUNT lines and number them, you should look into "nl -ba".

You could also do it all in awk:
awk '{ print $NR, $0 }' varlist.txt > out.txt

As Tim mentioned, you are reading EVERY line in your file, each time in your loop.

As Tim says you can use a read loop. Assuming you want to read all lines:
(( MYCOUNT = 1 ))
while read LINE; do
    echo "$MYCOUNT $LINE"
    (( MYCOUNT += 1 ))
done < varlist.txt > out.txt

[Amar_Joshi]

Thanks for your replies but it doesn't solve my problem.

DON'T look at the echo command, I just used it to illustrate the function, my actual command is 200 characters long and has lot's of other variables. Since the command is from hdvmcli.sh (storage utility) I hide it.

varlist.txt contains lots of lines looks like WWNs and my aim is to use them as one of the variables in line but when I use above mentioned example, $MYCOUNT increases but $CAID remains same (second line from the varlist.txt

Hope that is clear now....

[Dennis Handly]

>$MYCOUNT increases but $CAID remains same (second line from the varlist.txt

I believe we have told you why. Though it really should have ALL lines.

>Hope that is clear now.

You need to try my while read loop suggestion or say why it doesn't work.

[Peter Nikitka]

Hi,

this works, whether lines contain spaces or not:

OIFS="$IFS"
IFS=
typeset -i i=0
while read a
do print $((i+=1)) $a
done
Depending on your real needs it may be necessary to reset IFS to $oIFS in the loop.

mfG Peter

[James R. Ferguson]

Hi:

My thinking is the same as Dennis' and Peter. I would urge you to examine the manpages for 'read' and the 'sh-posix' pages that discuss the shell's IFS (Inter-Field-Seperator).

For example, if you wanted to read a file named 'varlist.txt' with whitespace as field delimiters, capturing in each iteration, the the *second* *through* *last* fields of each line in a variable named 'F2_N', you could do:

#!/usr/bin/sh
typeset -i MYCOUNT=0
while read F1 F2_N
do
(( MYCOUNT=MYCOUNT+1 ))
echo "${MYCOUNT} -> ${F2_N}" >> out.txt
done < varlist.txt

...Note that you need to write your output file (out.txt) in append mode (>>) to retain every iteration of the loop's output.

Regards!

...JRF...

[Peter Nikitka]

Hi,

an adding to JRF's solution:
You can possibly move the output redirection to the end of the loop, like
OIFS="$IFS"
IFS=
typeset -i i=0
while read a
do print $((i+=1)) $a
done outfile

Advantages:
- outfile needs not be cleared before the loop
- only one 'open' call instead of one per loop count

Additional output in the inner of the loop has to be written NOT to stdout, however.

mfG Peter

[James R. Ferguson]

Hi (again) Amardeep:

Peter's excellent ideas suggest another approach.

The code below accomodates:

1. The output file is automatically truncated (cleared) at the script's onset.

2. Only one open() call occurs.

3. *ALL* output in the loop is written to the output file regardless of the number of 'echo' or 'print' calls.

4. One argument can be optionally passed to the script to define the output file name. If no argument (filename) is provided, output is written to the current directory in a file named the same as the basename of the script with ".out" as a suffix.

# cat reader
#!/usr/bin/sh
typeset MYOUTPUT=${1:-$0.out}
typeset -i MYCOUNT=0
exec 1> ${MYOUTPUT}
while read F1 F2_N
do
(( MYCOUNT=MYCOUNT+1 ))
echo "${MYCOUNT} -> ${F2_N}"
done < varlist.txt

...when run as:

# ./reader

...output will be found in ./reader.out

Regards!

...JRF...

[Amar_Joshi]

Hi James, Peeter, Dennis.

Thanks for your suggestions. I finally got it working with Dennis' first suggestion of using "read LINE" option. While I tried others found this option is more easy to explain and easy to use.

Thanks again for all your help... you guys are just something more than a help, so called Gurus.