topic Re: Grep not functioning inside for/next or while loops. in Operating System - HP-UX

Grep not functioning inside for/next or while loops.

Johnny McKenzie — Wed, 01 Nov 2006 06:31:04 GMT

For reasons I don't understand the below grep statement does not generate output.

Outside a while or for/next loop, works fine, and inside the loop the shell knows of all the variables ( the echo statements )
But it simply will not generate output

If I change the $HOSTSFILE variable to an invalid directory, grep generates errors, so it is probally working, but I can't fathom where the output might be going. This is slowly driving me insane, while my project falls further behind. Any thoughts gratefully recieved

HP-UX B.11.00

#! /usr/bin/posix/sh

echo "$CONFIGCHG" | while read X ;do
echo "$X"; echo "$HOSTSFILE"
grep "$X" "$HOSTSFILE" | awk '{print $2}'
done

gives me this result:

$ storeconfig
138.27.120.94
/home/hynesm/bin/hosts
138.7.100.94
/home/hynesm/bin/hosts

echo statements work fine, but no result from the grep statement.
While directly on the command line, works fine.

$ HOSTSFILE="/home/hynesm/bin/hosts"
$ X=138.7.100.94
$
$ grep "$X" "$HOSTSFILE" | awk '{print $2}'
TST-RMC-2R1
$

Exactly what I would expect. Is grep working, and if so where is it sending it's output.

Re: Grep not functioning inside for/next or while loops.

Yang Qin_1 — Wed, 01 Nov 2006 06:45:19 GMT

Try:

#! /usr/bin/posix/sh

while read X
do
echo "$X"
echo "$HOSTSFILE"
grep "$X" "$HOSTSFILE" | awk '{print $2}'
done < $CONFIGCHG
exit

Yang

Re: Grep not functioning inside for/next or while loops.

Peter Godron — Wed, 01 Nov 2006 06:45:25 GMT

Hi,
what data is the while loop reading ?
Should the echo "$CONFIGCHG" be a cat "$CONFIGCHG"? Are there some semi-colons missing after echo "$HOSTSFILE".

Re: Grep not functioning inside for/next or while loops.

spex — Wed, 01 Nov 2006 07:33:48 GMT

Hi,

You probably want to 'cat $CONFIGCHG' rather than echo it. You can also combine the 'grep ... | awk ...' command into a single awk program:

awk "/$X/{print \$2}" $HOSTSFILE

PCS

Re: Grep not functioning inside for/next or while loops.

Hein van den Heuvel — Wed, 01 Nov 2006 07:45:15 GMT

- Missing semicolon?
- Do you need all those double quotes?
- Why not combine stuff?

I think the whoe script can be re-written as:

grep -f $CONFIGCHG $HOSTSFILE | awk '{print $2}'

And if you do need the loop for other reasons, then you can have AWK alone do the job. For example

awk -v x=$X '($1 ~ x){print $2}'

hth,
Hein.

Re: Grep not functioning inside for/next or while loops.

Johnny McKenzie — Wed, 01 Nov 2006 07:56:12 GMT

Thanks for your efforts guys, but still getting the same result.

Yang..I still get the grep mysterously not working.

Peter... the data is a list of IP address, as they are kept in a variable rather than a file, I beleive echo is the correct command
The semi colon takes place of a line break, as such I don't think it is required.

spex, I tried your awk command, and got exactly the same result..which is good as it means that the problem is not with grep or awk, but bad because I still don't have a clue while simple commands are not working.

Re: Grep not functioning inside for/next or while loops.

Yang Qin_1 — Wed, 01 Nov 2006 08:28:50 GMT

OK, if $CONFIGCHG is variable, then this one may work:

#! /usr/bin/posix/sh

for $X in $CONFIGCHG
do
echo "$X"
echo "$HOSTSFILE"
grep "$X" "$HOSTSFILE" | awk '{print $2}'
done

Yang

Re: Grep not functioning inside for/next or while loops.

Peter Godron — Wed, 01 Nov 2006 08:36:22 GMT

Hi,
change the $X to X in Yangs code and it works:
for $X in $CONFIGCHG
to
for X in $CONFIGCHG

Re: Grep not functioning inside for/next or while loops.

Johnny McKenzie — Wed, 01 Nov 2006 09:23:24 GMT

Spotted that, but it still dosn't work.
I appreciate the code is correct and it works on your machines, but there is obviously something mysterous going on with the box I'm working on.

As neither grep nor awk return results, the question is now:

"Why dosn't my shell show std output from inside loops ?"

Re: Grep not functioning inside for/next or while loops.

Yang Qin_1 — Wed, 01 Nov 2006 09:45:17 GMT

Let's "debug" your script. Can you add "set -x" on top of yor script and run it again. Or do add "set -x" but run script with sh -x <script_name>. Hope you will find something interesting.

Yang

Re: Grep not functioning inside for/next or while loops.

Christian Tremblay — Wed, 01 Nov 2006 10:01:38 GMT

A long shot, but it may be a path problem, try your script with full paths for commands.
/usr/bin/grep or /usr/bin/awk

Re: Grep not functioning inside for/next or while loops.

OldSchool — Wed, 01 Nov 2006 10:44:30 GMT

it appears that the "read" isn't parsing the ip addresses, so the grep is trying to match all of them at once. I get the same results.

the following appears to work as desired:

for X in `echo $CONFIGCHG`; do
echo ${X}
grep ${X} ${HOSTSFILE} | awk '{print $2}'
done

Re: Grep not functioning inside for/next or while loops.

Johnny McKenzie — Wed, 01 Nov 2006 11:14:55 GMT

I get the same result inside a for/next loop as with a read statement.

I tried the full paths, but with the same result.

I'm convinced this is something weird with the setup on the HP Box, and not with the code. All the examples provided should work.

I can't get anybody who looks after that box, to give me any time at all, so I'm thinking of going back to my orginal idea of using a LINUX/Bash enviroment.

Just have to convince my boss it's not "shareware" ( Corprates are weird, unless they spent thousands of quid on something, it's "worthless" )

Thanks for all your help, but this one is going into the "too hard" basket.

Re: Grep not functioning inside for/next or while loops.

OldSchool — Wed, 01 Nov 2006 12:09:13 GMT

Re-read the last batch of posts.

If CONFIGCHG contains more than one ip, for example:
123.234.123.1 123.234.222.111
it won't work with the following

echo $CONFIGCHK | while read X; do

the X gets the entire list, not the first IP. Changing it to

for X in $CONFIGCHK; do

appears to work as desired.....

Re: Grep not functioning inside for/next or while loops.

OldSchool — Wed, 01 Nov 2006 12:11:15 GMT

also, as noted above,

set -x at the beginning of your script to see how things are actually expanding.....

Re: Grep not functioning inside for/next or while loops.

Matti_Kurkela — Wed, 01 Nov 2006 12:13:44 GMT

Just one more thing to try.

Change your script to pipe the output of "echo $X" into "od" command with suitable options. Inspect for invisible characters, which might make it difficult for grep to find a match.

If the problem is there, re-think the way you're using to create the value of $CONFIGCHG.

Re: Grep not functioning inside for/next or while loops.

Sandman! — Wed, 01 Nov 2006 12:23:48 GMT

Can you run your script in debug mode by appending "-x" to the end of the first line and echo the value of $CONFIGCHG and post it here. i.e.

echo $CONFIGCHG

#! /usr/bin/posix/sh -x
.
.
.
rest of script
.
.
.

Re: Grep not functioning inside for/next or while loops.

Raynald Boucher — Thu, 02 Nov 2006 08:33:15 GMT

John,

try changeing your read statement to
"read X junk;".

I beleive the read command will load variables in sequence as they are contained in the line so that
read X makes X = the entire line
read X Y makes X = 1st word and Y = the rest of line
read X Y Z makes X = 1st, Y = 2nd and Z = rest of line.

RayB