topic Re: AWK Script Problem in Operating System - HP-UX

AWK Script Problem

Darren Etheridge_2 — Fri, 27 May 2005 10:11:41 GMT

I am trying to code a awk script (I'm a newbie) that will parse a file and spit the exceptions to a file; I can get it to display to the screen but not ONLY the exceptions listed in the if statement.. on a second note, if I have lots of values to compare in a if statement, is there a was to code a CASE statement in awk? Any help would be greatly appreciated and points will be awarded for all help :) below is my code:

cat /pei/peifas/ws/wsdata.dat |
awk 'BEGIN { FS=" "; }

value1 = substr($0, 1, 14)
gl_acct = 1234
#acct = substr($0, 4, 8)
damt = substr($0, 93, 13)
camt = substr($0, 105, 13)
desc = "W&S BATCH-" b_month""b_day""b_year
transamt = damt-camt
{
if (value1 = "4111151000000")
{ fund=411
acct=2071130

printf("||%s|%s||||%s||%s|%s|N|%s/%s/%s|WSIMPORT|WS%s%s%s|||\n", fund, acct, gl_
acct, desc, transamt, b_month, b_day, b_fullyear, b_month, b_day, b_year)
}

}'

Darren

Re: AWK Script Problem

Rodney Hills — Fri, 27 May 2005 10:24:29 GMT

Try changing your if statement to-

if (value1 == "411115100000")

Rather then using a case statement for multiple tests, you can use the the array syntax-
a["411115100000"]="411,2071130"
a["411222200000"]="511,3082240"
...
split(a[value1],b,",")
fund=b[1]
acct=b[2]
...

HTH

-- Rod Hills

Re: AWK Script Problem

Darren Etheridge_2 — Fri, 27 May 2005 10:29:07 GMT

How would I kick the true values of the if statement to a file?

Re: AWK Script Problem

Darren Etheridge_2 — Fri, 27 May 2005 10:31:09 GMT

BTW, attached is my data file

Re: AWK Script Problem

H.Merijn Brand (procura — Fri, 27 May 2005 10:32:20 GMT

You need perl. First, you know awk, so, the way to learn perl from awk is to use a2p (awk to perl translator)

Simplified output from a2p

--8<---
#!/usr/bin/perl

$[ = 1; # set array base to 1

$FS = ' ';

while (<>) {
chomp; # strip record separator
print $_ if $value1 = substr($_, 1, 14);
print $_ if $gl_acct = 1234;
#acct = substr($0, 4, 8)
print $_ if $damt = substr($_, 93, 13);
print $_ if $camt = substr($_, 105, 13);
print $_ if $desc = 'W&S BATCH-' . $b_month . '' . $b_day . '' . $b_year;
print $_ if $transamt = $damt - $camt;

if ($value1 = '4111151000000') {
$fund = 411;
$acct = 2071130;

printf "||%s|%s||||%s||%s|%s|N|%s/%s/%s|WSIMPORT|WS%s%s%s|||\n",

$fund, $acct, $gl_acct, $desc, $transamt, $b_month, $b_day,

$b_fullyear, $b_month, $b_day, $b_year;
}
}
-->8---

I know perl much better than awk, and looking at that code, it's probably not what you want

I also don't see b_month, b_day, b_year, and b_fullyear being defined anywhere

In the if statement, you are using = instead of ==, causing an asignment instead of a comparison

--8<---
#!/usr/bin/perl

while (<>) {
chomp;

my ($b_day, $b_month, b_year) = (localtime)[3,4,5];
my $b_fullyear = $b_year + 1900;

my $gl_acct = 1234;
my ($value1, $damt, $camt) = unpack "A14 x79 A13 A13", $_;
my $transamt = $damt - $camt;
my $desc = sprintf "W&S BATCH-%4d%02d%02d", $b_fullyear, $b_month + 1, $b_day;

$value1 eq "4111151000000" or next;

my $fund = 411;
my $acct = 2071130;

printf "||%s|%s||||%s||%s|%s|N|%s/%s/%s|WSIMPORT|WS%s%s%s|||\n",
$fund, $acct, $gl_acct, $desc, $transamt, $b_month, $b_day, $b_fullyear, $b_month, $b_day, $b_year;
}
-->8---

Do my comments make any sense?

Enjoy, Have FUN! H.Merijn

Re: AWK Script Problem

Rodney Hills — Fri, 27 May 2005 10:34:47 GMT

A typical way to select certain data and print to a file would be-

awk 'if (substr($0,0,4) == "abcd") { print $0 }' outputfile

By redirecting STDOUT to a file, then the lines will be printed to a file...

If you want to get more complicated, like several output files, then you can add "> file" to the print statement.

HTH

-- Rod Hills

Re: AWK Script Problem

H.Merijn Brand (procura — Fri, 27 May 2005 10:43:44 GMT

Just to make sure I got the numbers right, I took the file from your other post and tested

--8<--- 246551.pl
#!/usr/bin/perl

use strict;
use warnings;

while (<>) {
my ($b_day, $b_month, $b_year) = (localtime)[3,4,5];
my $b_fullyear = $b_year + 1900;

my $gl_acct = 1234;
my ($value1, $damt, $camt) = unpack "A14 x79 A13 A11", $_;
#print;
#print "$value1|$damt|$camt|\n";
my $transamt = $damt - $camt;
my $desc = sprintf "W&S BATCH-%4d%02d%02d", $b_fullyear, $b_month + 1, $b_day;

$value1 eq "4111151000000" or next;

my $fund = 411;
my $acct = 2071130;

printf "||%s|%s||||%s||%s|%s|N|%s/%s/%s|WSIMPORT|WS%s%s%s|||\n",
$fund, $acct, $gl_acct, $desc, $transamt,
$b_month, $b_day, $b_fullyear, $b_month, $b_day, $b_year;
}
-->8---

lt09:/tmp 117 > perl 246551.pl 246551.dat
||411|2071130||||1234||W&S BATCH-20050527|-16.48|N|4/27/2005|WSIMPORT|WS427105|||
lt09:/tmp 118 >

Enjoy, Have FUN! H.Merijn

Re: AWK Script Problem

Gopi Sekar — Fri, 27 May 2005 10:44:24 GMT

ok the problem seems to be simple:

for any statement which is not covered by brackets, awk prints the entire record. so to fix your problem

start '{' after BEGIN block.

eg:

awk 'BEGIN { FS=" "; "
{
value1 = substr
...

}'

also the if condition value being used is one character short in length in the dat file you have given. check that too.

as far as i know there is no switch case functionality in awk.

Regards,
Gopi

Re: AWK Script Problem

Jean-Luc Oudart — Fri, 27 May 2005 10:56:29 GMT

Darren,

If you are only interested in "4111151000000" at the beginning of your record you should code a grep and pipe into awk :

grep ^"4111151000000" /pei/peifas/ws/wsdata.dat | awk '{
...
}'
also the test will be
if(value1=="...")

Regards
Jean-Luc

Re: AWK Script Problem

Hein van den Heuvel — Fri, 27 May 2005 11:31:28 GMT

[ 0 points for this nitpicking/debate reply ]

I respectfully disagree with Jean-Luc's suggestion to filter through grep first.

AWK is good at, and happy to, look for the match itself? Why shove the data through grep, into a pipe, into awk?

The same applies to Darren's suggested use of 'cat' to read the file and feed it to awk.

Awk is perfectly happy to read the file itself! Furthermore, awk (or perl) will have the actual filename available which may be useful for more elaborte scripts with error handlers.

And a field seperator of " " is default.
So the basic awk syntax for this problem is simply:

awk '/^4111151000000/ { fund=... print...}' pei/peifas/ws/wsdata.dat

fwiw,
Hein.

Re: AWK Script Problem

Darren Etheridge_2 — Fri, 27 May 2005 11:43:25 GMT

Well this part of my script works perfectly. Now in my input file I have about 100 different account to compare and change values to match our system. What is the best way to compare each value instead of an if then else structure.. since I can't use CASE, in addition I want the values that don't meet any of the comparisions (TRUE) to be kicked to a dump file. BTW, Gopi your answer was KEY to fix my problem... THANKS TO ALL!!!!

cat ws.datx >> wsdata.dat
echo "Parsing wsdata.dat" >> $LOGFL

cat /pei/peifas/ws/wsdata.dat |
awk 'BEGIN { FS=" "; x = 0; }
{
if ( x == 0 ){
b_month = substr($0,1,2)
b_day = substr($0,3,2)
b_fullyear = substr($0,5,4)
b_year = substr($0,7,2)
x = 1
} else {

value1 = substr($0, 1, 14)
gl_acct = 1234
#acct = substr($0, 4, 8)
damt = substr($0, 93, 13)
camt = substr($0, 105, 13)
desc = "W&S BATCH-" b_month""b_day""b_year
transamt = damt-camt
{
if (value1 == "41122010110296")
{ fund="411"
acct="2071130"

printf("||%s|%s||||%s||%s|%s|N|%s/%s/%s|WSIMPORT|WS%s%s%s|||\n", fund, acct, gl_acct, desc, transamt, b_month, b_day, b_fullyear, b_month, b_day, b_year)
}}
}
}' > ws.out

Re: AWK Script Problem

H.Merijn Brand (procura — Fri, 27 May 2005 11:56:02 GMT

[ /me shamelessly keeps plugging perl as *the* solution to these growing problems ]

To continue on what I had

somewhere in the beginning you now will have

my %hash = (
# value => [ fund, acct ]
4111151000000 => [ 411, 2071130 ],
4213253710000 => [ 421, 2098812 ],
:
:
);

then where I had the test,

exists $hash{$value1} or next;
my ($fund, $acct) = @{$hash{$value1}};

printf "||%s|%s||||%s||%s|%s|N|%s/%s/%s|WSIMPORT|WS%s%s%s|||\n",
$fund, $acct, $gl_acct, $desc, $transamt,
$b_month, $b_day, $b_fullyear, $b_month, $b_day, $b_year;

and you don't need no switch/case at all!

Enjoy, Have FUN! H.Merijn

Re: AWK Script Problem

Rodney Hills — Fri, 27 May 2005 12:25:53 GMT

My first post showed how to do it with "awk" using "associative arrays". It is similar to what Procura demonstrated. If you text processing gets much more complicated, then "perl" would be the more prefered tool.

My 2 cents

-- Rod Hills

Re: AWK Script Problem

Rodney Hills — Fri, 27 May 2005 12:33:36 GMT

Procura,

I'm thought you might have changed-
printf "||%s|%s||||%s||%s|%s|N|%s/%s/%s|WSIMPORT|WS%s%s%s|||\n",
$fund, $acct, $gl_acct, $desc, $transamt,
$b_month, $b_day, $b_fullyear, $b_month, $b_day, $b_year;

to use join-
print join("|","","",$fund, $acct, "","","",$gl_acct,"", $desc, $transamt,"N",join("/",$b_month, $b_day, $b_fullyear), "WS".$b_month.$b_day.$b_year,"","",""),"\n";

Which I think is more readable then a printf...

My 4 cents...

-- Rod Hills

Re: AWK Script Problem

Darren Etheridge_2 — Fri, 27 May 2005 13:36:35 GMT

Rodney,
I tried using the array syntax you advised but can't seem to get it to only process the TRUE values of the array... How would I do this?

transamt = damt-camt

#if (value1 == "41122010110296")
# { fund="411"
# acct="2071130"
{
a["41122010110296"]="411,2071130"

split(a[value1],b,",")

fund=b[1]
acct=b[2]

printf("||%s|%s||||%s||%s|%s|N|%s/%s/%s|WSIMPORT|WS%s%s%s|||\n", fund, acct, gl_
acct, desc, transamt, b_month, b_day, b_fullyear, b_month, b_day, b_year)

}
}'

Re: AWK Script Problem

Rodney Hills — Fri, 27 May 2005 13:47:35 GMT

You will need one if statement.

BEGIN{
a["411115100000"]="411,2071130"
a["411222200000"]="511,3082240"
}
{
value1=substr($0,1,14)
...
if (a[value1]) {
split(a[value1],b,",")
fund=b[1]
acct=b[2]
print value1,fund,acct
}
}

If a[value1] does not have a value, which means you have not set a predefined mapping in the BEGIN block, then the if block will not print the record.

HTH

-- Rod Hills

Re: AWK Script Problem

H.Merijn Brand (procura — Fri, 27 May 2005 14:18:59 GMT

Rodney, yes, I usualy do not use printf for this, but in this phase of plugging perl to take over awk, I think I should not push the limits. Pushing the limits, would be using $,, $", and "@{[join]}" just for obfuscational reasons

I also think that with so many fields, csv is to be prefered over '|' separators

Enjoy, Have FUN! H.Merijn