Operating System - Linux
1748022 Members
5019 Online
108757 Solutions
New Discussion юеВ

Re: Doing Sum of Colums with awk

 
SOLVED
Go to solution
Hein van den Heuvel
Honored Contributor

Re: Doing Sum of Colums with awk


Hello Chris,

>> I still get the "cannot have more than 199 fields"

I don't have access to an HPUX box to try this just now. Is that just a warning and it continues or a hard failure?
Awk on windows does not fail, just does not give access to fields over $99

>> I was thinking of zipping the file and attaching it so that you could see that I am not changing any columns.

And that looks much cleaner!

>> I am using textpad to read the file and not notepad as notepad is a bit different.

I'm also using textpad.(and sometimes crimson)
Since you are loooking at the data using windows, how about processing it on windows!?

>> What am I doing wrong.

You are using standard awk which is primitive. You need to learn awk better.. if you are going to use it. You probably should be using perl, the data is 'ugly'. Other than that all is well ! :-).

The main part of the script now uses 'substr' to select fields. It still uses $1 and $2 for free format input. Replace that by substr and then you can call awk with -Fx giving a non-whitespace seperator and it will no longer see all those fields. Up to you to do the column counting!


Cheers,
Hein.
Peter Godron
Honored Contributor

Re: Doing Sum of Colums with awk

Hi,

>> I still get the "cannot have more than 199 fields"

Have you forgotten my inital mail ?
From "man awk" :
" DIAGNOSTICS
awk supports up to 199 fields ($1, $2, ..., $199) per record."

Thats on my HPUX 11.11 box.
Chris Frangandonis
Regular Advisor

Re: Doing Sum of Colums with awk

Hi Gordon,

Firstly thank you and yes you are right. Question on gawk
1) Which gawk version and is the site from http://ftp.gnu.org/gnu/gawk/
2) Can Hein's awk script be accommodated to use gawk? I am not to familiar with gawk

Many Thanks
Chris
Peter Godron
Honored Contributor

Re: Doing Sum of Colums with awk

Chris,
http://hpux.connect.org.uk/hppd/hpux/Gnu/gawk-3.1.5/
or
http://mirrors.develooper.com/hpux/

The script should pretty much run as is, but I am not an awk/gawk expert.

Perhaps one of experts has found a way around the 199 field limit ...?
Chris Frangandonis
Regular Advisor

Re: Doing Sum of Colums with awk

Hi Hein,

Are you familiar with gawk and can you script run with gawk.

Thanks
Chris
Hein van den Heuvel
Honored Contributor

Re: Doing Sum of Colums with awk

Sorry, can not try anything on hpux just now.
I'm out at a customer site and did not set up my rx2620 for remote access this time.
The 199 error *might* be coming from the debug statement 'print NF'. If so you are in luck. Just delete that line.

Hein.
Chris Frangandonis
Regular Advisor

Re: Doing Sum of Colums with awk

Hi Hein,Peter,

I installed gwak on the HPUX and that did the trick. So far 100% done. Thanks Peter.

Hein,
One last and final question:
I need to include the digit before Data ID i.e.
06-11-2600:45"1" 7 198
{ if (NR > 1) {
line = $0
len = length($0)
pos = 14 # Changed from 15 to 14
adder = 10 # first size is exclusive of type/size
while (len - pos > 10) {
$0 = substr (line,pos+1,20)
etc......
reason being :
DATA_Quality
=============
0 DEFAULT default value
1 SECURE data of this integration interval are secure
2 TIME CORR during this cycle the system time was changed by the MML command
3 NO INIT counter could not become initialized
If the value (DATA_Q) is " 0 || 1 " then continue adding or else skip line (dont take into consideration (No adding is necessary).

I must admit Hein, your idea is brilliant, great stuff. I could learn a lot from you.

Thanks Again
Chris



Hein van den Heuvel
Honored Contributor
Solution

Re: Doing Sum of Colums with awk

You could change that first line to explicitly implement the formula in little steps:

{ q = substr ($0,14,1)
if ((NR > 1) and ((q == "1") or (q =="0")) ) {
:

But I would fix it by using the standard awk regular expression filter technique. Change the first line with the IF to read:
/^..-..-....:..[01]/ {

and drop a line with "}" before the END section.

This makes the main section conditional on the line starting with (^) that date pattern, the (:), two anythings and a 0 or 1 ([01]).


>> I must admit Hein, your idea is brilliant, great stuff. I could learn a lot from you.

Thank you.
(For mere money I'll come over and teach some tricks and methods! I'm a gun for hire, consulting primarily in software/database/application performance space :-)

Regards,
Hein van den Heuvel
HvdH Performance Consulting
Chris Frangandonis
Regular Advisor

Re: Doing Sum of Colums with awk

Hein,

Brilliant stuff. I used the search "/..-[[01]/ and it works. Magic.
One last favor, if I ever need some help which I know I would, will it be possible to mail you my questions. It will be questions once a blue moon (No nagging) and if so please post or mail me your address.

Many Thanks and keep up the good stuff
Chris
Hein van den Heuvel
Honored Contributor

Re: Doing Sum of Colums with awk

It's in my profile, or google for hein hvdh
[0 points for this please :-]

Hein.