Grep not functioning inside for/next or while loops.

[Johnny McKenzie]

For reasons I don't understand the below grep statement does not generate output.

Outside a while or for/next loop, works fine, and inside the loop the shell knows of all the variables ( the echo statements )
But it simply will not generate output

If I change the $HOSTSFILE variable to an invalid directory, grep generates errors, so it is probally working, but I can't fathom where the output might be going. This is slowly driving me insane, while my project falls further behind. Any thoughts gratefully recieved

HP-UX B.11.00

#! /usr/bin/posix/sh

echo "$CONFIGCHG" | while read X ;do
echo "$X"; echo "$HOSTSFILE"
grep "$X" "$HOSTSFILE" | awk '{print $2}'
done

gives me this result:

$ storeconfig
138.27.120.94
/home/hynesm/bin/hosts
138.7.100.94
/home/hynesm/bin/hosts

echo statements work fine, but no result from the grep statement.
While directly on the command line, works fine.

$ HOSTSFILE="/home/hynesm/bin/hosts"
$ X=138.7.100.94
$
$ grep "$X" "$HOSTSFILE" | awk '{print $2}'
TST-RMC-2R1
$

Exactly what I would expect. Is grep working, and if so where is it sending it's output.

[Yang Qin_1]

Try:

#! /usr/bin/posix/sh

while read X
do
echo "$X"
echo "$HOSTSFILE"
grep "$X" "$HOSTSFILE" | awk '{print $2}'
done < $CONFIGCHG
exit


Yang

[Peter Godron]

Hi,
what data is the while loop reading ?
Should the echo "$CONFIGCHG" be a cat "$CONFIGCHG"? Are there some semi-colons missing after echo "$HOSTSFILE".

[spex · Accepted solution]

Hi,

You probably want to 'cat $CONFIGCHG' rather than echo it. You can also combine the 'grep ... | awk ...' command into a single awk program:

awk "/$X/{print \$2}" $HOSTSFILE

PCS

[Hein van den Heuvel]

- Missing semicolon?
- Do you need all those double quotes?
- Why not combine stuff?

I think the whoe script can be re-written as:

grep -f $CONFIGCHG $HOSTSFILE | awk '{print $2}'


And if you do need the loop for other reasons, then you can have AWK alone do the job. For example

awk -v x=$X '($1 ~ x){print $2}'

hth,
Hein.

[Johnny McKenzie]

Thanks for your efforts guys, but still getting the same result.

Yang..I still get the grep mysterously not working.

Peter... the data is a list of IP address, as they are kept in a variable rather than a file, I beleive echo is the correct command
The semi colon takes place of a line break, as such I don't think it is required.

spex, I tried your awk command, and got exactly the same result..which is good as it means that the problem is not with grep or awk, but bad because I still don't have a clue while simple commands are not working.

[Yang Qin_1]

OK, if $CONFIGCHG is variable, then this one may work:

#! /usr/bin/posix/sh

for $X in $CONFIGCHG
do
echo "$X"
echo "$HOSTSFILE"
grep "$X" "$HOSTSFILE" | awk '{print $2}'
done


Yang

[Peter Godron]

Hi,
change the $X to X in Yangs code and it works:
for $X in $CONFIGCHG
to
for X in $CONFIGCHG

[Johnny McKenzie]

Spotted that, but it still dosn't work.
I appreciate the code is correct and it works on your machines, but there is obviously something mysterous going on with the box I'm working on.

As neither grep nor awk return results, the question is now:

"Why dosn't my shell show std output from inside loops ?"

[Yang Qin_1]

Let's "debug" your script. Can you add "set -x" on top of yor script and run it again. Or do add "set -x" but run script with sh -x <script_name>. Hope you will find something interesting.

Yang

[Christian Tremblay]

A long shot, but it may be a path problem, try your script with full paths for commands.
/usr/bin/grep or /usr/bin/awk

[OldSchool]

it appears that the "read" isn't parsing the ip addresses, so the grep is trying to match all of them at once. I get the same results.

the following appears to work as desired:

for X in `echo $CONFIGCHG`; do
echo ${X}
grep ${X} ${HOSTSFILE} | awk '{print $2}'
done

[Johnny McKenzie]

I get the same result inside a for/next loop as with a read statement.

I tried the full paths, but with the same result.

I'm convinced this is something weird with the setup on the HP Box, and not with the code. All the examples provided should work.

I can't get anybody who looks after that box, to give me any time at all, so I'm thinking of going back to my orginal idea of using a LINUX/Bash enviroment.

Just have to convince my boss it's not "shareware" ( Corprates are weird, unless they spent thousands of quid on something, it's "worthless" )

Thanks for all your help, but this one is going into the "too hard" basket.

[OldSchool]

Re-read the last batch of posts.

If CONFIGCHG contains more than one ip, for example:
123.234.123.1 123.234.222.111
it won't work with the following

echo $CONFIGCHK | while read X; do

the X gets the entire list, not the first IP. Changing it to

for X in $CONFIGCHK; do

appears to work as desired.....

[OldSchool]

also, as noted above,

set -x at the beginning of your script to see how things are actually expanding.....

[Matti_Kurkela]

Just one more thing to try.

Change your script to pipe the output of "echo $X" into "od" command with suitable options. Inspect for invisible characters, which might make it difficult for grep to find a match.

If the problem is there, re-think the way you're using to create the value of $CONFIGCHG.

[Sandman!]

Can you run your script in debug mode by appending "-x" to the end of the first line and echo the value of $CONFIGCHG and post it here. i.e.

echo $CONFIGCHG

#! /usr/bin/posix/sh -x
.
.
.
rest of script
.
.
.

[Raynald Boucher]

John,

try changeing your read statement to
"read X junk;".

I beleive the read command will load variables in sequence as they are contained in the line so that
read X makes X = the entire line
read X Y makes X = 1st word and Y = the rest of line
read X Y Z makes X = 1st, Y = 2nd and Z = rest of line.

RayB