topic Re: Awk Problem in Operating System - HP-UX

Awk Problem

Chris Frangandonis — Tue, 23 Jul 2002 18:45:43 GMT

Hi All,

I have a file in hex which looks as follows :

0000000 4844 5231 6466 3233 4241 4430 3232 3030
0000010 0800 4040 07fc 0000 c6c4 8472 1002 0200
0000020 2211 1500 0000 0000 00ff ff02 4040 4040
0000030 4040 4040 4040 4040 0100 2601 1101 0200
0000040 1755 2400 003a 0000 0103 ff02 0102 0209
0000050 010a 0103 2191 3256 8f05 0601 0293 0471
0000060 1f01 0027 0111 0102 0017 5527 0001 7400
0000070 0002 03ff 0201 0202 0901 0f01 0321 9053
0000080 227f 0507 0102 0114 2442 9501 0027 0111
0000090 0102 0017 5527 0002 eb00 0004 03ff 0201
00000a0 0202 0901 0a01 0321 9512 584f 0507 0102
00000b0 0837 4651 9901 0027 0111 0102 0017 5528
00000c0 0001 2000 0000 03ff 0201 0202 0901 0f01
00000d0 0321 9020 005f 0507 0102 0800 0068 7001

By using the awk command or any other, I would the output to be in separate lines for eg:

0100 2601 1101 0200 1755 2400 1755 2400 003a 0000 0103 ff02 0102 0209 010a 0103 2191 3256 8f05 0601 0293 0471 1f01
01 0027 0111 0102 0017 5527 0001 7400 0002 03ff 0201 0202 0901 0f01 0321 9053 227f 0507 0102 0114 2442 95
01 0027 0111 etc.....
The first coloum ($1) should not appear.

Kind Regards
Chris

Re: Awk Problem

Hai Nguyen_1 — Tue, 23 Jul 2002 18:55:28 GMT

This should work:

# cut -d" " -f2-9 input_file

Assuming that there are 9 fields per line with a single space as delimiter between any two fields.

Hai

Re: Awk Problem

James R. Ferguson — Tue, 23 Jul 2002 18:58:48 GMT

Hi Chris:

If you want to drop the first field of every record, outputting the remainder, you could do:

# awk '{for (i=2;i
Regards!

...JRF...

Re: Awk Problem

Rich Wright — Tue, 23 Jul 2002 19:04:22 GMT

With the "cut" solution, you can just use.
cut -d" " -f2-

This will not limit you to 9 fields.

Re: Awk Problem

A. Clay Stephenson — Tue, 23 Jul 2002 19:10:35 GMT

The various solutiopn will work but you seem to front-ending the file with od. You must do that because cut, awk, sed, etc. will not be able to handle input that is not LF delimted.

I assume that you have chosen the desired -t x conversion. I also suggest that you use od -v so that repeated lines are displayed explicitly.

Re: Awk Problem

A. Clay Stephenson — Tue, 23 Jul 2002 19:26:03 GMT

By the way, you are aware that you could specify a -An od argument and suppress the offset column. That's the easy way to do this.

Re: Awk Problem

Chris Frangandonis — Tue, 23 Jul 2002 19:27:16 GMT

Hi Hai,JRF

Thanks for your input, maybe I did not phrase it correctly.
What I need is, by looking at the sample file in hex, (as above) scan the file until 0100 26 (could be 01 0027 or 0100 28, it changes as goes on) for eg which is the first record and converting it to dec = 38 bytes long, so the output should look as follows

0100 2601 1101 0200 1755 2400 003a 0000 0103 ff02 0102 0209 010a 0103 2191 3256 8f05 0601 0293 0471 1f,
which is equal to 38 bytes long counting from 0100 26. Please note that it's not always 38 bytes long, it could vary +- and again not taken the first coloumn into account.

Once again
Thanks
Chris

Re: Awk Problem

Rodney Hills — Tue, 23 Jul 2002 19:28:49 GMT

I'm not sure how-

0100 2601 1101 0200 1755 2400 1755 2400 003a 0000 0103 ff02 0102 0209 010a 0103 2191 3256 8f05 0601 0293 0471 1f01
01 0027 0111 0102 0017 5527 0001 7400 0002 03ff 0201 0202 0901 0f01 0321 9053 227f 0507 0102 0114 2442 95
01 0027 0111 etc.....

relates to your original file. Do you want 23 numbers grouped per line?

Maybe more elaboration on how the resulting data should look would help.

-- Rod Hills

Re: Awk Problem

James R. Ferguson — Tue, 23 Jul 2002 19:41:40 GMT

Hi (again):

You should look at 'xd' (also known as 'od'). As Clay noted, there are numerous options to 'xd'. You can request the output in decimal, skip bytes, etc. Note that 'xd' will read from standard-in so, of course, redirection from some other file or pipe will work!

Regards!

...JRF...

Re: Awk Problem

Chris Frangandonis — Tue, 23 Jul 2002 19:42:08 GMT

Hi All,

Many thanks for your input
I have attached a file which is colour format, maybe this would help.

Re: Awk Problem

Rodney Hills — Tue, 23 Jul 2002 19:43:41 GMT

If this is a binary file with a record "layout", then I would recommend creating a perl script to read the file directly. By writing a subroutine that would do the logical record reads, you could easily manipulate the file.

-- Rod Hills

Re: Awk Problem

Chris Frangandonis — Tue, 23 Jul 2002 19:51:58 GMT

Hi Rod,

The answer to your question is "No", its in hex

Thanks
Chris

Re: Awk Problem

Rodney Hills — Tue, 23 Jul 2002 20:18:22 GMT

In your attachment you indicated you were using "xd". Then you do have a binary file (xd displays data in a hex format).

The following perl script could be used to read the orginal file (not the result of the xd).

open(INP,"seek INP,0x38,0; # position past header
while(read(INP,$recsiz,3)) { # get record size
($b1,$rsiz)=unpack("c1,c2",$recsiz);
if (read(INP,$datum,$rsiz)) {
$l=len($datum)-1;
for $i (0..$l) {
$c=substr($datum,$i,1);
printf "%1.1x",$c;
print " " if ($i % 2) != 0;
}
}
}

-- Rod Hills

Re: Awk Problem

Rodney Hills — Tue, 23 Jul 2002 20:26:25 GMT

oops, forgot to a line in the code. Insert

print "\n";

before the last 2 "}".
This way it will display a logical record in hex per text line of print.

-- Rod Hills

Re: Awk Problem

Chris Frangandonis — Wed, 24 Jul 2002 16:03:10 GMT

Hi Ron, all

Thanks for the input. I have run yor script but came back with errors "cant open file at .Open_perl line 3", so I changed the fileinput to its orig filename and still error "Undefined subroutine $main :: len called at .Open_perl line 9". I am at a learning stage of perl.
I know that awk can do the trick and have read a lot about it. Harry d Brown once helped me with a similar problem, but nowhere to be found. Still looking for an answer

Many Thanks
Chris

Re: Awk Problem

harry d brown jr — Wed, 24 Jul 2002 16:16:27 GMT

Chris,

Can you post your script, so that we can see the changes you made?

thanks,

live free or die
harry

Re: Awk Problem

H.Merijn Brand (procura — Wed, 24 Jul 2002 16:18:36 GMT

'perl -v' is not showing 4.018 or something I hope?

Re: Awk Problem

Chris Frangandonis — Wed, 24 Jul 2002 16:35:39 GMT

Hi Harry,

Thanks once again. Once I can put the records in separate lines then the script will follow.
You once helped me with "reverse byte in hex" with conv to decimal.

echo 1234 2345 3456 8e00 | awk '
{
hexstr="0123456789abcdef"
newnum=tolower(substr($4,3,2) substr($4,1,2))
slen = length(newnum)
decnum = 0
for (i=1; i < slen+1;i++)
{
tpos = index(hexstr,substr(newnum,i,1)) - 1
decnum = decnum + ( (16 ** (slen -i)) * tpos)
}
printf("%s is %d\n",newnum,decnum)
} '

Please look at my attachment.

Many Thanks
Chris

Re: Awk Problem

H.Merijn Brand (procura — Thu, 25 Jul 2002 07:17:50 GMT

I cannot look at a .doc attachment. Please use .txt, .sh (shar), .tgz (tar/gzip) or other readable unix formats.

The script you show is awk, but you reported perl problems. A quick browse of that awk shows that whatever you are doing can be done much more simple in perl.

Re: Awk Problem

Rodney Hills — Thu, 25 Jul 2002 13:27:04 GMT

With out the binary file, I can't fully test the script. But I fixed my original to correct syntax problems.

Again use the perl script on the original binary file (not the "xd" output file). Replace "fileinput" with the name of the binary file.

open(INP,"seek INP,0x38,0; # position past header
while(read(INP,$recsiz,3)) { # get record size
($b1,$rsiz)=unpack("c1,c2",$recsiz);
if (read(INP,$datum,$rsiz)) {
# uncomment next line if you want record size to be displayed too
# $datum=$recsiz . $datum;
$l=length($datum)-1;
for $i (0..$l) {
$c=substr($datum,$i,1);
printf "%1.1x",ord($c);
print " " if ($i % 2) != 0;
}
print "\n";
}
}

Hope this works better.

-- Rod Hills