how to use RS in awk

[Gary Yu]

dear all,

how can I specify a record separater(RS) as a full string. I mean, how can I use a line of string to be RS, so that each record is a multi-line block. the following command works fine in Linux
awk -v RS='NEW ERROR MESSAGE' '{print NR}' my_log_file
but in HPUX, it seemed only the first character 'N' is taken as RS, so it break the record at everywhere there's a 'N'

so how can I set RS as a full line?
thanks,
Gary

[Chris Wilshaw]

Try running

awk -F"New Error Message" '{print NR}' my_log_file


From the awk man page;

-F fs Specify regular expression used to separate fields.

[Gary Yu]

Hi Chris,

thanks for the reply.

I think -F'fs' is to separate fields, what I need is to separated records. e.g I have a file with 3000 lines, and the pattern is, for every 5 lines, there's a line 'NEW ERROR MESSAGE', so I want to process these 5 lines as one single record, in other words, the NR I get should be 600, not 3000.

I tried your command, still get 3000
thanks,
Gary

[Muthukumar_5]

hai,

awk scans and process each input file as line by line. We can not use the FS to limit it for 5 lines.

We have to do the shell script to do this.

Regards,
Muthukumar.

[Gary Yu]

Hi Muthukumar,

Actually, as said before, I found with awk on HP, it can process record of multi-lines, only problem is it only takes the first character I set with "RS" as the separater instead of the whole string.

with the 3000 lines example I said before, it breaks into records more than 600, but definitely less than 3000, by taking a closer look, I found it broke down the records whereever there's 'N'

but this commad works fine on Linux :(

[Muthukumar_5]

Make a simple script to do this as,

#!/usr/bin/ksh
set -x

# Usage: <script>
filename=$1

> /tmp/testforum.log

getline()
{
input=$1
number=$2

echo $input >> /tmp/testforum.log

if [[ $((line%5)) -eq 0 ]]; then
cat /tmp/testforum.log
> /tmp/testforum.log
fi

}

index=1

while read line; do

getline $line $index

let index=index+1

done < $filename

Regards,
Muthukumar.

[Chris Wilshaw]

How about

grep -c "NEW ERROR MESSAGE" my_log_file

This gives the count of the message separators in the file - slightly easier than awk.

[Gary Yu]

and please note, I'm using RS here, not FS

thanks,
Gary

[Gary Yu]

thanks again Muthukumar and Chris,

Muthukumar, your script looks great, but the reason I wantto use awk, is it's more flexible and simple, say, what if the pattern is not strictly fixed line numbers, and I can only separate each record by the string 'NEW ERROR MESSAGE', not by line number(which is actually most of the case), and I need to do some more stuff within each record, thus I can't use grep only.

Gary

[Massimo Bianchi]

Hi,
mabe this can help ?

awk '/NEW ERROR MESSAGE/,/$/ { print NR }' etc_etc_Etc

REgards,
Massimo

[Abdul Rahiman]

If I understand it correctly,
Put this in as a awk script, called myawk,
BEGIN {
FS="\n"
RS="MY ERROR"
}
{
print NR
}
and run the script against the file,
# awk -f myawk logfile

regds,
Abdul.

[curt larson_1]

just another work around

pick a character that isn't likely to be in your log file. insert that character before your string to use as a record seperator. then use that character as your record seperator

char="^V"
cat logfile |
awk -v char="$char" '
/^NEW ERROR MESSAGE/ {print char $0;next;}
{print;}' |
awk -v RS="$char" '{print NR;}'

and if you don't like the blank record at the begining do something like this

awk -v char="$char" '
/^NEW ERROR MESSAGE/ {if (num>0) {print char $0;next;}num++}
{print;}' |

[Gary Yu]

Abdul,

I'm not sure it's my version of awk of something, but awk only recognise the first character of RS, which is 'N' in my example and 'M' (as in MY ERROR) in your example

thanks,
Gary

[Hein van den Heuvel · Accepted solution]

My Suse Linux 2.4.21 box indeed allows for a lng string as record seperator.

However, my Tru64 Unix system behaves just like hpux, so this is not behaviour to rely on / expect.
It just happened to work.

Here is a possible workaround:

awk '{x = x RS $0} /^NEW ERROR MESSAGE$/ {nr++; print x; x=""} END {print x; print nr}' my_log_file

hth,
Hein.

[Abdul Rahiman]

Gary,

It looks like the awk implementation on unix doesn't allow string as a field separator.
A work around would be to convert the record separator string to a unique character which is not in the file and do the awk operation on the output.
eg,
replacing the string to "|" and changing the RS="|" in the myawk script,
# sed "s/MY ERROR/\|/" abc |awk -f awk-rs

regds,
Abdul.