topic Re: adding numbers don't add up.. in Operating System - HP-UX

adding numbers don't add up..

Dave Chamberlin — Wed, 18 Jul 2001 15:20:51 GMT

I have a small C program that adds .01 to a number X times.
If the number is 1000, the result is 10.00, for 10000 the result is 100.00 but for higher numbers, the result is not correct (x=100000 gives 1000.67, x=1000000 gives 9865.22). What gives?

Re: adding numbers don't add up..

Sachin Patel — Wed, 18 Jul 2001 15:32:04 GMT

Hi Dave,
try double or long I am not sure. It is been long time.

long flot x;

Sachin

Re: adding numbers don't add up..

Kenneth Platz — Wed, 18 Jul 2001 15:34:28 GMT

Dave,

What datatype are you using for your additions? If you're currently using a "float", then you may want to try using a "double" or "long double", because those have greater precision.

I hope this helps.

Re: adding numbers don't add up..

Rodney Hills — Wed, 18 Jul 2001 15:34:50 GMT

Due to the way floating point numbers are represented, the concept of precision is taken with a grain of salt.

If you wish to calculate to exactly 2 decimal places, then scale an integer number (ie instead of 1.23 of "real" type, use 123 in integer). Thus in calculations your results will be precise. Convert to "real" only for printing.

Re: adding numbers don't add up..

Andreas Voss — Wed, 18 Jul 2001 15:51:25 GMT

Hi,

setting

double f=0.0;

is sufficient.

Regards