Awk Problem

[Chris Frangandonis]

Hi All,

I have a file in hex which looks as follows :

0000000 4844 5231 6466 3233 4241 4430 3232 3030
0000010 0800 4040 07fc 0000 c6c4 8472 1002 0200
0000020 2211 1500 0000 0000 00ff ff02 4040 4040
0000030 4040 4040 4040 4040 0100 2601 1101 0200
0000040 1755 2400 003a 0000 0103 ff02 0102 0209
0000050 010a 0103 2191 3256 8f05 0601 0293 0471
0000060 1f01 0027 0111 0102 0017 5527 0001 7400
0000070 0002 03ff 0201 0202 0901 0f01 0321 9053
0000080 227f 0507 0102 0114 2442 9501 0027 0111
0000090 0102 0017 5527 0002 eb00 0004 03ff 0201
00000a0 0202 0901 0a01 0321 9512 584f 0507 0102
00000b0 0837 4651 9901 0027 0111 0102 0017 5528
00000c0 0001 2000 0000 03ff 0201 0202 0901 0f01
00000d0 0321 9020 005f 0507 0102 0800 0068 7001

By using the awk command or any other, I would the output to be in separate lines for eg:

0100 2601 1101 0200 1755 2400 1755 2400 003a 0000 0103 ff02 0102 0209 010a 0103 2191 3256 8f05 0601 0293 0471 1f01
01 0027 0111 0102 0017 5527 0001 7400 0002 03ff 0201 0202 0901 0f01 0321 9053 227f 0507 0102 0114 2442 95
01 0027 0111 etc.....
The first coloum ($1) should not appear.

Kind Regards
Chris

[Hai Nguyen_1]

This should work:

# cut -d" " -f2-9 input_file

Assuming that there are 9 fields per line with a single space as delimiter between any two fields.

Hai

[James R. Ferguson]

Hi Chris:

If you want to drop the first field of every record, outputting the remainder, you could do:

# awk '{for (i=2;i
Regards!

...JRF...

[Rich Wright]

With the "cut" solution, you can just use.
cut -d" " -f2-

This will not limit you to 9 fields.

[A. Clay Stephenson]

The various solutiopn will work but you seem to front-ending the file with od. You must do that because cut, awk, sed, etc. will not be able to handle input that is not LF delimted.

I assume that you have chosen the desired -t x conversion. I also suggest that you use od -v so that repeated lines are displayed explicitly.

[A. Clay Stephenson]

By the way, you are aware that you could specify a -An od argument and suppress the offset column. That's the easy way to do this.

[Chris Frangandonis]

Hi Hai,JRF

Thanks for your input, maybe I did not phrase it correctly.
What I need is, by looking at the sample file in hex, (as above) scan the file until 0100 26 (could be 01 0027 or 0100 28, it changes as goes on) for eg which is the first record and converting it to dec = 38 bytes long, so the output should look as follows

0100 2601 1101 0200 1755 2400 003a 0000 0103 ff02 0102 0209 010a 0103 2191 3256 8f05 0601 0293 0471 1f,
which is equal to 38 bytes long counting from 0100 26. Please note that it's not always 38 bytes long, it could vary +- and again not taken the first coloumn into account.

Once again
Thanks
Chris

[Rodney Hills]

I'm not sure how-

0100 2601 1101 0200 1755 2400 1755 2400 003a 0000 0103 ff02 0102 0209 010a 0103 2191 3256 8f05 0601 0293 0471 1f01
01 0027 0111 0102 0017 5527 0001 7400 0002 03ff 0201 0202 0901 0f01 0321 9053 227f 0507 0102 0114 2442 95
01 0027 0111 etc.....

relates to your original file. Do you want 23 numbers grouped per line?

Maybe more elaboration on how the resulting data should look would help.

-- Rod Hills

[James R. Ferguson]

Hi (again):

You should look at 'xd' (also known as 'od'). As Clay noted, there are numerous options to 'xd'. You can request the output in decimal, skip bytes, etc. Note that 'xd' will read from standard-in so, of course, redirection from some other file or pipe will work!

Regards!

...JRF...

[Chris Frangandonis]

Hi All,

Many thanks for your input
I have attached a file which is colour format, maybe this would help.

[Rodney Hills]

If this is a binary file with a record "layout", then I would recommend creating a perl script to read the file directly. By writing a subroutine that would do the logical record reads, you could easily manipulate the file.

-- Rod Hills

[Chris Frangandonis]

Hi Rod,

The answer to your question is "No", its in hex

Thanks
Chris

[Rodney Hills]

In your attachment you indicated you were using "xd". Then you do have a binary file (xd displays data in a hex format).

The following perl script could be used to read the orginal file (not the result of the xd).

open(INP,"seek INP,0x38,0; # position past header
while(read(INP,$recsiz,3)) { # get record size
($b1,$rsiz)=unpack("c1,c2",$recsiz);
if (read(INP,$datum,$rsiz)) {
$l=len($datum)-1;
for $i (0..$l) {
$c=substr($datum,$i,1);
printf "%1.1x",$c;
print " " if ($i % 2) != 0;
}
}
}

-- Rod Hills

[Rodney Hills]

oops, forgot to a line in the code. Insert

print "\n";

before the last 2 "}".
This way it will display a logical record in hex per text line of print.

-- Rod Hills

[Chris Frangandonis]

Hi Ron, all

Thanks for the input. I have run yor script but came back with errors "cant open file at .Open_perl line 3", so I changed the fileinput to its orig filename and still error "Undefined subroutine $main :: len called at .Open_perl line 9". I am at a learning stage of perl.
I know that awk can do the trick and have read a lot about it. Harry d Brown once helped me with a similar problem, but nowhere to be found. Still looking for an answer

Many Thanks
Chris

[harry d brown jr]

Chris,

Can you post your script, so that we can see the changes you made?

thanks,

live free or die
harry

[H.Merijn Brand (procura]

'perl -v' is not showing 4.018 or something I hope?

[Chris Frangandonis]

Hi Harry,

Thanks once again. Once I can put the records in separate lines then the script will follow.
You once helped me with "reverse byte in hex" with conv to decimal.

echo 1234 2345 3456 8e00 | awk '
{
hexstr="0123456789abcdef"
newnum=tolower(substr($4,3,2) substr($4,1,2))
slen = length(newnum)
decnum = 0
for (i=1; i < slen+1;i++)
{
tpos = index(hexstr,substr(newnum,i,1)) - 1
decnum = decnum + ( (16 ** (slen -i)) * tpos)
}
printf("%s is %d\n",newnum,decnum)
} '

Please look at my attachment.

Many Thanks
Chris

[H.Merijn Brand (procura]

I cannot look at a .doc attachment. Please use .txt, .sh (shar), .tgz (tar/gzip) or other readable unix formats.

The script you show is awk, but you reported perl problems. A quick browse of that awk shows that whatever you are doing can be done much more simple in perl.

[Rodney Hills]

With out the binary file, I can't fully test the script. But I fixed my original to correct syntax problems.

Again use the perl script on the original binary file (not the "xd" output file). Replace "fileinput" with the name of the binary file.

open(INP,"seek INP,0x38,0; # position past header
while(read(INP,$recsiz,3)) { # get record size
($b1,$rsiz)=unpack("c1,c2",$recsiz);
if (read(INP,$datum,$rsiz)) {
# uncomment next line if you want record size to be displayed too
# $datum=$recsiz . $datum;
$l=length($datum)-1;
for $i (0..$l) {
$c=substr($datum,$i,1);
printf "%1.1x",ord($c);
print " " if ($i % 2) != 0;
}
print "\n";
}
}

Hope this works better.

-- Rod Hills

[Chris Frangandonis]

Hi Rod,all

Once again thanks.
I have attached the original binary file. I hope you can read it.
One question, why would you prefer to use perl rather than unix? The reason I question this, is due to me being more profecient on unix more than perl.


Many Thanks
Chris

[Tom Maloy]

Chris,

I assume that you mean the Unix shells and/or tools like sed, awk, ... Vendors are also distributing (slightly older) versions of perl with their OS, recognizing that perl is a very powerful tool.

Use whatever tool works best for the application. If you investigate perl, you will be pleasantly surprised by its power and flexibility.

Tom

[Chris Frangandonis]

Hi Harry,

Any luck with to my question.

Chris

[Rodney Hills]

The attachment you provided can not be opened because it is binary. Process the file through uuencode and attach that resultant file.

-- Rod Hills

[Chris Frangandonis]

Hi Rod,

I have attached the file in uuencoded format.

Chris