topic Re: File interrogation in Operating System - HP-UX

File interrogation

cbres00 — Wed, 26 Feb 2003 00:49:24 GMT

Friends,
I have a file named
ABCxxxx.txt where xxxx changes but ABC remains constant.

Within file ABCxxxx.txt I should have the following fixed layout:
aa,xxxx,bb,cccc where the second column should always match the xxxx digits in the file name.

What's the most efficient way to:
a) test the 'xxxx' in the filename against the 'xxxx' in every row of the file? (I know how to get the xxxx out of the filename; it's the testing against the records I'm unsure of);

b) How can I toss out those records that don't have a match?

Looking forward to your input!
Cath

Re: File interrogation

cbres00 — Wed, 26 Feb 2003 00:50:50 GMT

One more thing: in addition to tossing out records that don't match, I need to preserve the originals in a different file.

CB

Re: File interrogation

Sridhar Bhaskarla — Wed, 26 Feb 2003 01:04:05 GMT

Hi Cath,

#!/usr/bin/ksh
FILE=ABCXXXX.txt
MYNAME=$(echo $FILE|awk -F "." '{print $1}'|sed 's/ABC//g')

awk -v STR="$MYNAME" -F "," '$2 == STR {print $0}' $FILE > $FILE.new

-Sri

Re: File interrogation

Sridhar Bhaskarla — Wed, 26 Feb 2003 01:06:25 GMT

Hi (Again),

The above will create a new file called ABCXXXX.txt.new. You will still have the original file preserved. Replace ABCXXXX.txt with your file

-Sri

Re: File interrogation

cbres00 — Wed, 26 Feb 2003 04:22:33 GMT

I got the following error after executing the following:
awk -v STR=0139 -F , $2 == STR {print $0} ABC1390.txt

"awk: syntax error near line 1"

Re: File interrogation

Sridhar Bhaskarla — Wed, 26 Feb 2003 04:48:52 GMT

Hi,

You changed the syntax of the awk statement. Copy and Paste the above script into a file - call it script. Replace ABCXXXX.txt with ABC1390.txt and run the script. It will produce ABC1390.txt.new.

Or run the command

awk -F "," '$2 == "0139" {print $0}' ABC1390.txt

will work for you if you have only one such file.

-Sri

Re: File interrogation

cbres00 — Wed, 26 Feb 2003 05:20:26 GMT

Sri,
I figured out that my error was putting quotes around the comma separator. When I do -F, it works.

So the errors go to $FILE.new.
Where would the good data land? I need to end up with a file of good data and a file of bad data (if indeed there is bad data!) And you KNOW there's bad data in this world!

Cathy

Re: File interrogation

cbres00 — Wed, 26 Feb 2003 08:08:14 GMT

I'm at a loss. I set up four files, one of which has the incorrect xxxx in the second column. I am not getting those records excluded into the new file.

Any other ideas?

Re: File interrogation

john korterman — Wed, 26 Feb 2003 10:00:59 GMT

Hi Cathy,
could you please attach some authentic input containing good and bad records?

regards,
John K.

Re: File interrogation

cbres00 — Wed, 26 Feb 2003 16:00:50 GMT

Good file:
File ABC0041.txt:
05,0041,20030226,15

Bad file:
File ABC1346.txt
10,0139,20030226,15

I need to test the second column in the file make sure it matches the four digits in the file name. If the data's good it should go to one file; if it's bad it should go to another.

Thanks!
Cathy

Re: File interrogation

Sridhar Bhaskarla — Wed, 26 Feb 2003 16:44:45 GMT

Hi Cathy,

Here is the modified script. Now do not run the individual commands as you may run into syntax issues. Just modify ABCXXXX.txt to match your filename and then run the script as it is. This will generate two files. ABCXXXX.txt.good and ABCXXXX.txt.bad.

----copy below---

#!/usr/bin/ksh
FILE=ABC1390.txt
MYNAME=$(echo $FILE|awk -F "." '{print $1}'|sed 's/ABC//g')

awk -v STR="$MYNAME" -F "," '$2 == STR {print $0}' $FILE > $FILE.good
awk -v STR="$MYNAME" -F "," '$2 != STR {print $0}' $FILE > $FILE.bad

----till here---

-Sri

Re: File interrogation

cbres00 — Wed, 26 Feb 2003 17:25:05 GMT

Hi Sri.
I finally got it to work but only by using nawk rather than awk.

Thanks for your valuable assistance. Enjoy the points!

Cathy