IEEE Floating Point Problems

[Joseph DiCara]

I have an application writen in C. The application is complied and linked to use IEEE floating point. The enclosed text file is from a debug session. Can any one explain what I am doing wrong.

[Stanley F Quayle]

It may be late in the middle of the night, but I don't see any problem. Can you be more explicit?

[Joseph DiCara]

All three calculations should have the same result. The first answer calculates to 101.355010986328. I then multiply all the numbers (except the divisor by 100) and the result is 10135.5000000000. If the first calculation was correct,then the second answer should have been 10135.5010986328. In the last step. I multiply the divsior by 100 and the result is 101.355000000000. Which is what the first calculation should have returned as the answer.

The question is "Why is the first calculation wrong ?".

[Stanley F Quayle]

I'd be willing to bet that if you assigned each of the individual numbers to variables, and looked at their values, one or more won't be represented exactly. You're thinking in terms of decimal, while the computer is binary -- it's sort of the difference between 1/3 and .333333.

You might look at double-precision floating-point, although I'm not sure how to do that in the debugger. That should make the "error" different, but still not zero.

[Guy Peleg]

I'd like to know what is it that you are trying
to fix. Do you have problems in runtime or
only in the debugger?

From your output it looks like you are using
VMS V7.1, most of the fixes for IEEE related
bugs are not available on V7.1

Guy