awk decimal -> hex conversion?

[Doug O'Leary]

Riddle me this, batman!

On all of our 11.11 systems:

# echo "" | awk {printf("%x\n", 1119965980)}'
42c1531c # Correct:
# echo "" | awk '{printf("%x\n", 2482526102)}'
7fffffff # Incorrect;

On 11.23 systems, it works correctly:
echo "" | awk '{printf("%x\n", 2482526102)}'
93f85796 # Correct:

Even on my linux laptop:
$ echo "" | awk '{printf("%x\n", 2482526102)}'
93f85796

All kernels are 64 bit where appropriate (obviously not on my linux laptop...) I tried installing PHCO_29952 w/no success. Curiously enough, printf works:

# printf "%x\n" 2482526102
93f85796

Anyone know what might be going on? Thanks.

Doug

[Pete Randall]

Doug,

I can confirm that all 6 of my 11.11 systems came up with exactly the same incorrect answer as yours. Odd, indeed.


Pete

[Patrick Wallek]

It appears that somewhere there is a limit.

7fffffff = 2,147,483,647 (or 2^31 - 1)

which is 1 byte short of 2GB (2^31) if one were talking storage space.

If you do your test with:

echo "" | awk '{printf("%x\n",2147483646)'

you get 7ffffffe which is correct.

I can't explain why it is happening or how to fix it though.

[RAC_1]

I am thinking that this is coming out of sheel limit of doing arithmatic calculations.
If I remember correctly, the shell can not do
anything more than 247xxxxxxxx somthing.
Which shell are you using.

[Patrick Wallek]

Is your 11.23 system Itanium or PA-RISC? I just tried the same on my HP-UX 11.23 PA-RISC and I see the same exact behavior as on my 11.11 system (2147483649 is shown as 7fffffff instead of 80000001).

[Marvin Strong]

On Itanium 11.23 server it does work correctly.
with awk.

It does work with printf from within ksh or a posix shell. On an 11.11 machine.

# ksh
# printf "%x\n" 2482526102
93f85796

This kinda looks like a awk bug. where it cant handle a number larger than. 2147483647

[A. Clay Stephenson]

Whether or not this work will depend upon the native size of an int. You should really plan on everything having the 32-bit limit and always use bc for this kind of conversion if overflow is even remotely possible. Bc supports unlimited precision and size.

[Doug O'Leary]

Hey, all;

Thanks for the replies. I was thinking it had to do with some kind of limit as well; however, it's not the shell's limit as can be proven by observing the printf statement directly.

I was hoping someone would say "Oh yea; apply this patch". That doesn't look like it's going to happen though.

Thanks again for the responses.

Doug O'Leary

[Bill Hassell · Accepted solution]

Indeed, shells only perform interger arithmetic, 32 bits only, regardless of what the kernel is doing. The kernel never sees any of this and it's important to isolate what pieces are handled by the shell and by processes. So in assigning a number to a variable, you can certainly do something like this:

x=2482526102
echo $x
2482526102

but force the variable to be integer only:

typeset x=2482526102
echo $x
-1812441194

This is because untyped variables start as strings and if used in arithmetic calculations, the equivalent integer value is used. The best way to avoid problems like this is to perform the calculations in bc (and you can convert decimal, octal, hex, etc) and be careful about using shell variables for calculations. If the variable is used as a string and passed to bc, the results will be accurate.