topic Using division operator produces negative values in Operating System - HP-UX

Using division operator produces negative values

Gilbert Standen_1 — Wed, 23 Jun 2004 17:22:06 GMT

following code is used in my ksh script and for large non-negative positive integers it is returning "bizarre" negative integer values. What am I missing here ?

let bytes_K=$bytes/1024

for example,
bytes_K = 4194304000
and output of code snippet is:
-98304
???

Re: Using division operator produces negative values

curt larson_1 — Wed, 23 Jun 2004 17:25:42 GMT

my first guess would be your running into the limitation for 32 integer numbers

see what you get for numbers less then 2G and then for numbers over 2G

you'll probably have to use something like bc or perl to handle such large numbers

Re: Using division operator produces negative values

Steven E. Protter — Wed, 23 Jun 2004 17:36:50 GMT

Value field overflowed. Your number is too big for the field. I'm wondering why it never happened to me.

Maybe it did?

:-)

SEP

Re: Using division operator produces negative values

Sundar_7 — Wed, 23 Jun 2004 17:42:53 GMT

use bc -l

echo "$bytes/1024" | bc -l

Re: Using division operator produces negative values

Gilbert Standen_1 — Wed, 23 Jun 2004 17:43:42 GMT

The replies are greatly appreciated!
Got around it by doing my units conversion to Kbytes while within the Oracle engine, but glad to know there was a valid issue there. I'll have to find a solid piece of code one of these days soon for handling division of large integers!
Gil

Re: Using division operator produces negative values

A. Clay Stephenson — Wed, 23 Jun 2004 17:45:12 GMT

Run this command to display the range of integers supported:

getconf LONG_MIN
getconf LONG_MAX

If your vales are outside this range then you need a plan B.

One simple solution is "bc".

Here is an an example:

#!/usr/bin/sh

kbytes()
{
typeset -i10 STAT=0
typeset N=${1}
shift
echo "scale=0; ${N} / 1024" | bc
STAT=${?}
return ${STAT}
} # kbytes

BYTES=4194304000
KBYTES=$(kbytes ${BYTES})
echo "Result = ${KBYTES} KB"

The "scale=0" set 0 decimal places in the result; "scale=2" would set 2 decimal places"

Man bc for more details. Bc can also be used for floating point calculations -- something that the shell will not do.

Re: Using division operator produces negative values

Gilbert Standen_1 — Wed, 23 Jun 2004 17:58:54 GMT

Apparently the problem is arising at the "let x=' stmt as well as the arithmetic computation code, e.g.

422 let x=5
423 echo $x
5
424 let x=2000
425 echo $x
2000
426 let x=4194304000
427 echo $x
-100663296

now if I do

429 getconf LONG_MIN
-2147483648
430 getconf LONG_MAX
2147483647

Now if I do:
432 echo "$x/1024" | bc -l
-98304.00000000000000000000

I'll try out your expanded solution and see if it works for me...it's appreciated!

Re: Using division operator produces negative values

Gilbert Standen_1 — Wed, 23 Jun 2004 18:06:35 GMT

Mr. Stephenson, your solution is excellent! I'm sure I shall have cause to use it many times. It is much appreciated.