topic IEEE Floating Point Problems in Operating System - OpenVMS

IEEE Floating Point Problems

Joseph DiCara — Fri, 16 May 2003 13:32:54 GMT

I have an application writen in C. The application is complied and linked to use IEEE floating point. The enclosed text file is from a debug session. Can any one explain what I am doing wrong.

Re: IEEE Floating Point Problems

Stanley F Quayle — Sat, 17 May 2003 06:12:20 GMT

It may be late in the middle of the night, but I don't see any problem. Can you be more explicit?

Re: IEEE Floating Point Problems

Joseph DiCara — Mon, 19 May 2003 12:06:46 GMT

All three calculations should have the same result. The first answer calculates to 101.355010986328. I then multiply all the numbers (except the divisor by 100) and the result is 10135.5000000000. If the first calculation was correct,then the second answer should have been 10135.5010986328. In the last step. I multiply the divsior by 100 and the result is 101.355000000000. Which is what the first calculation should have returned as the answer.

The question is "Why is the first calculation wrong ?".

Re: IEEE Floating Point Problems

Stanley F Quayle — Mon, 19 May 2003 12:19:01 GMT

I'd be willing to bet that if you assigned each of the individual numbers to variables, and looked at their values, one or more won't be represented exactly. You're thinking in terms of decimal, while the computer is binary -- it's sort of the difference between 1/3 and .333333.

You might look at double-precision floating-point, although I'm not sure how to do that in the debugger. That should make the "error" different, but still not zero.

Re: IEEE Floating Point Problems

Guy Peleg — Mon, 19 May 2003 13:40:30 GMT

I'd like to know what is it that you are trying
to fix. Do you have problems in runtime or
only in the debugger?

From your output it looks like you are using
VMS V7.1, most of the fixes for IEEE related
bugs are not available on V7.1

Guy