topic Re: parse a file with expressions (grep) in Operating System - HP-UX

parse a file with expressions (grep)

Billa-User — Fri, 01 Oct 2010 09:53:18 GMT

hello,

i have to change a existing shell script.

here my issue:

demo file (field separator or ): test.cnf
entry1 entry2
entry1 entry2 var

shell script should find following:
search with : entry1 and entry2 => OK
but should not find
entry1 entry2 var
search with : entry1 and entry2 and var => OK
search with : entry1 and entry2 and notvar => Not-OK

# shell script test.sh
# begin
# if doesn't work when field separator is
entry1=$1
entry2=$2

var=$3

if grep "^[ \t]*${entry1}[ \t]" test.cnf >/dev/null
then
if grep "^${entry1}[ \t]*${entry2}$" test.cnf >/dev/null
then
if grep "^${entry1}[ \t]*${entry2}
[ \t]*${entry2}${var}$" test.cnf >/dev/null
then
echo "entry1 : ${entry1} entry2: ${entry2} var:${var} "
fi
else
echo "entry1 : ${entry1} entry2: ${entry2}"
fi
fi
fi

# begin

regards

Re: parse a file with expressions (grep)

Jean-Luc Oudart — Fri, 01 Oct 2010 10:32:31 GMT

Hi

may be you could use awk. Pass the 3 variables to the awk script. Check $1 and $2 vs entry1 and entry2. if you have a 3rd field (check NF for nb field in current record), the decide to print the relevant result.

awk -v en1=$entry1 -v en2=$entry2 -v var=$var
'{if($1==en1) { if($2==en2)
{ ...
}
' test.cnf

Regards
Jean-Luc

Re: parse a file with expressions (grep)

Hein van den Heuvel — Fri, 01 Oct 2010 11:45:52 GMT

I would like perl or awk for this also, as one program activation can do it all versus the convuleted nesting.

As you indeed looking for 3 specific words with option whitespace?

If you need further help, then you may want to reply with a TEXT file attachment showing sample input lines which should, or should not match.

Speaking of which, it is not immediately clear to me if/how you want to deal with multiple matching or partially matching lines.

Using awk you can test for the presence of 'var' using NF = 3 or NF > 2

Anyway... It looks like the 'whitespace' requirement simple was not carried out completely.

Specifically, the second and third grep read: grep "^${entry1}...
Should that not be: grep "^[ \t]*${entry1}

And on the 3rd grep it reads: ${entry2}${var}
So those two words should be adjacent without whitespace?

hth,
Hein

Re: parse a file with expressions (grep)

Laurent Menase — Fri, 01 Oct 2010 11:57:45 GMT

why not

grep -v "$entry1 $entry2 "

Re: parse a file with expressions (grep)

Dennis Handly — Sat, 02 Oct 2010 05:15:53 GMT

Instead of "grep foo file > /dev/null" you can simply this as:
grep -q foo file

>but should not find: entry1 entry2 var

Are you going to check for an empty string for $var?

Do you have an example of your script, your datafile and your script parms?

>[ \t]

This escape "\t" doesn't work for grep.

Re: parse a file with expressions (grep)

Billa-User — Mon, 04 Oct 2010 05:30:41 GMT

parameters for testing:

./test.sh entry1 entry2
./test.sh entry1 entry2 var

the test script should parse file "test.cnf" (attachment)
field separator or .

regards

Re: parse a file with expressions (grep)

Billa-User — Mon, 04 Oct 2010 05:31:26 GMT

test script:

test.sh

Re: parse a file with expressions (grep)

Billa-User — Mon, 04 Oct 2010 10:50:09 GMT

hello,

i tested following "awk" and it matches the exact entries (field separator or ), is it a good solution ?

awk -v entry1="${entry1}" -v entry2="${entry2}" -v var="${var}" 'BEGIN { FS = "[ \t]*|[ \t]+" }
$1 ~ /^'entry1'$/ && $2 ~ /^'entry2'$/ && $3 ~ /^'var'$/ { print "FOUND" }' test.cnf

if [ `awk -v entry1="${entry1}" -v entry2="${entry2}" -v var="${var}" 'BEGIN { FS = "[ \t]*|[ \t]+" }
$1 ~ /^'entry1'$/ && $2 ~ /^'entry2'$/ && $3 ~ /^'var'$/ { print "FOUND" }' test.cnf` = "FOUND" ]
then
echo "awk: FOUND"
fi

also a test with "grep" :
OLDIFS=$IFS
IFS="[ \t]*|[ \t]+"
grep "^${entry1} ${entry2} ${var}$" test.cnf

IFS=$OLDIFS

Re: parse a file with expressions (grep)

James R. Ferguson — Mon, 04 Oct 2010 11:27:22 GMT

Hi:

> i tested following "awk" and it matches the exact entries (field separator or ), is it a good solution ?

OK, but for the 'grep' code, you haven't paid attention to what Dennis said about the use of '\t' for a TAB character: HP's 'grep' doesn't recognize it and you are being fooled into thinking it does when you use

# grep "^[ \t]*"

Consider that this matches:

# echo "aaa"|grep "^[\t ]*"
aaa

You can match because the asterisk ("*") matches ZERO or more characters. You have none in the example above, and that *meets* the criteria.

Further, the '\t' isn't understood by 'grep' in HP-UX. If it was, this would work:

# echo "\t"|grep "[\t]"

...and it does *not*. You could use a literally composed TAB (which the Forums formatting will obliterate unless you copy-and-paste this:

# echo "\t"|grep "[ ]"

AWK *does* understand the '\t' notation, however.

If you use 'grep -E' to invoke the extended regular expression engine, you can use '+' to signify ONE or more instances of the preceding character.

AWK supports extended regular expresssions, so this can be used there, too.

Regards!

...JRF...

Re: parse a file with expressions (grep)

Dennis Handly — Mon, 04 Oct 2010 11:53:35 GMT

>I tested following "awk" and it matches the exact entries (field separator or ), is it a good solution?

Since this is the default, you don't want to set FS.
And if you are going to use awk, you shouldn't have to invoke it more than once.

>also a test with grep:
>IFS="[ \t]*|[ \t]+"
>grep "^${entry1} ${entry2} ${var}$" test.cnf

grep doesn't look at IFS, only the shell and you don't have the proper format for IFS since it normally has: space, tab, and newline

Re: parse a file with expressions (grep)

Dennis Handly — Tue, 05 Oct 2010 02:19:37 GMT

>awk -v entry1="${entry1}" -v entry2="${entry2}" -v var="${var}" 'BEGIN { FS = "[ \t]*|[ \t]+" }
$1 ~ /^'entry1'$/ && $2 ~ /^'entry2'$/ && $3 ~ /^'var'$/ { print "FOUND" }' test.cnf

There is no need for your single quotes around entry1, etc. They aren't doing what you think.
Also, there may be no need to do pattern matching, you could do exact match:
awk -v entry1="${entry1}" -v entry2="${entry2}" -v var="${var}" '
$1 == entry1 && $2 == entry2 && $3 == var { print "FOUND" }' test.cnf

And if you really wanted to do pattern matching you would need to use something like:
$1 ~ /^'$entry1'$/
(And forget about using awk -v.)

Or do:
$1 ~ "^" entry1 "$"

Re: parse a file with expressions (grep)

Billa-User — Tue, 05 Oct 2010 13:20:12 GMT

hello,

thx to all for your input, last i have a special question, i add to "test.cnf" following entry:

* * var

and start test.sh (in attachment)

./test.sh entry1 entry2 var

this should find:
entry1 entry2 var
* * var

how can i use "*" in awk ?

regards,

Re: parse a file with expressions (grep)

Dennis Handly — Wed, 06 Oct 2010 10:19:42 GMT

>this should find: * * var
>how can I use "*" in awk?

Where are your patterns? On the command line or in your test.cnf?

Do you want to have the command line or test.cnf have EREs? Or do you want an exact match with a special case of "*" in test.cnf?

>if ($1 == /^\*/ && entry2 == /^\*/) {

I have no idea what this will do. == should take either a literal string, number or variable. I guess "/^\*/" is just an expression and it seems to have the value 0 if it doesn't match $0.

Re: parse a file with expressions (grep)

Billa-User — Thu, 07 Oct 2010 10:19:55 GMT

@ Where are your patterns? On the command line or in your test.cnf?
in the attachment script (test.sh)

@ Do you want to have the command line or test.cnf have EREs? Or do you want an exact match with a special case of "*" in test.cnf?
i want it , if it easy to handle in test.cnf.

sorry, it explained it very bad .

what i mean :

i want to make an entry in config-file with wildcard's? like :

* * var

and then i want to search with a statement all entry's field 1 and 2 and with exact value in field 3 (example : var)

also i think it complicated to search entries with example
entry* entry* var

but this i don't need for my case.

in the meantime in think about following,
i create a placeholder or wildcard for "*" like "allmatches" and it is
easier to handle in statement's . example:

allmatches allmatches var

Re: parse a file with expressions (grep)

Dennis Handly — Sat, 09 Oct 2010 01:45:13 GMT

>I want to make an entry in config-file with wildcards? like: * * var
>I want to search with a statement all entries field 1 and 2 and with exact value in field 3 (example: var)

>I think it complicated to search entries with example: entry* entry* var
>but this I don't need for my case.

Right, this is harder but if you use EREs, you can do that in awk.

>I create a placeholder or wildcard for "*" like "allmatches" and it is easier to handle in statements

No, "*" is easier:
# Pattern in test.cnf
# Use == for match
awk -v entry1="${entry1}" -v entry2="${entry2}" -v var="${var}" '
($1 == "*" || $1 == entry1) &&
($2 == "*" || $2 == entry2) &&
$3 == var { print "FOUND5:", $0 }' test.cnf

Re: parse a file with expressions (grep)

Billa-User — Mon, 11 Oct 2010 06:21:32 GMT

perfect,thank you very,very much!!