topic Re: awk decimal -> hex conversion? in Operating System - HP-UX

awk decimal -> hex conversion?

Doug O'Leary — Mon, 15 Aug 2005 13:24:00 GMT

Riddle me this, batman!

On all of our 11.11 systems:

# echo "" | awk {printf("%x\n", 1119965980)}'
42c1531c # Correct:
# echo "" | awk '{printf("%x\n", 2482526102)}'
7fffffff # Incorrect;

On 11.23 systems, it works correctly:
echo "" | awk '{printf("%x\n", 2482526102)}'
93f85796 # Correct:

Even on my linux laptop:
$ echo "" | awk '{printf("%x\n", 2482526102)}'
93f85796

All kernels are 64 bit where appropriate (obviously not on my linux laptop...) I tried installing PHCO_29952 w/no success. Curiously enough, printf works:

# printf "%x\n" 2482526102
93f85796

Anyone know what might be going on? Thanks.

Doug

Re: awk decimal -> hex conversion?

Pete Randall — Mon, 15 Aug 2005 13:29:55 GMT

Doug,

I can confirm that all 6 of my 11.11 systems came up with exactly the same incorrect answer as yours. Odd, indeed.

Pete

Re: awk decimal -> hex conversion?

Patrick Wallek — Mon, 15 Aug 2005 13:46:32 GMT

It appears that somewhere there is a limit.

7fffffff = 2,147,483,647 (or 2^31 - 1)

which is 1 byte short of 2GB (2^31) if one were talking storage space.

If you do your test with:

echo "" | awk '{printf("%x\n",2147483646)'

you get 7ffffffe which is correct.

I can't explain why it is happening or how to fix it though.

Re: awk decimal -> hex conversion?

RAC_1 — Mon, 15 Aug 2005 13:51:47 GMT

I am thinking that this is coming out of sheel limit of doing arithmatic calculations.
If I remember correctly, the shell can not do
anything more than 247xxxxxxxx somthing.
Which shell are you using.

Re: awk decimal -> hex conversion?

Patrick Wallek — Mon, 15 Aug 2005 13:52:14 GMT

Is your 11.23 system Itanium or PA-RISC? I just tried the same on my HP-UX 11.23 PA-RISC and I see the same exact behavior as on my 11.11 system (2147483649 is shown as 7fffffff instead of 80000001).

Re: awk decimal -> hex conversion?

Marvin Strong — Mon, 15 Aug 2005 14:08:47 GMT

On Itanium 11.23 server it does work correctly.
with awk.

It does work with printf from within ksh or a posix shell. On an 11.11 machine.

# ksh
# printf "%x\n" 2482526102
93f85796

This kinda looks like a awk bug. where it cant handle a number larger than. 2147483647

Re: awk decimal -> hex conversion?

A. Clay Stephenson — Mon, 15 Aug 2005 15:33:05 GMT

Whether or not this work will depend upon the native size of an int. You should really plan on everything having the 32-bit limit and always use bc for this kind of conversion if overflow is even remotely possible. Bc supports unlimited precision and size.

Re: awk decimal -> hex conversion?

Doug O'Leary — Mon, 15 Aug 2005 19:26:07 GMT

Hey, all;

Thanks for the replies. I was thinking it had to do with some kind of limit as well; however, it's not the shell's limit as can be proven by observing the printf statement directly.

I was hoping someone would say "Oh yea; apply this patch". That doesn't look like it's going to happen though.

Thanks again for the responses.

Doug O'Leary

Re: awk decimal -> hex conversion?

Bill Hassell — Mon, 15 Aug 2005 19:46:55 GMT

Indeed, shells only perform interger arithmetic, 32 bits only, regardless of what the kernel is doing. The kernel never sees any of this and it's important to isolate what pieces are handled by the shell and by processes. So in assigning a number to a variable, you can certainly do something like this:

x=2482526102
echo $x
2482526102

but force the variable to be integer only:

typeset x=2482526102
echo $x
-1812441194

This is because untyped variables start as strings and if used in arithmetic calculations, the equivalent integer value is used. The best way to avoid problems like this is to perform the calculations in bc (and you can convert decimal, octal, hex, etc) and be careful about using shell variables for calculations. If the variable is used as a string and passed to bc, the results will be accurate.