HPE Community read-only access December 15, 2018
This is a maintenance upgrade. You will be able to read articles and posts, but not post or reply.
Hours:
Dec 15, 4:00 am to 10:00 am UTC
Dec 14, 10:00 pm CST to Dec 15, 4:00 am CST
Dec 14, 8:00 pm PST to Dec 15, 2:00 am PST
Operating System - OpenVMS
cancel
Showing results for 
Search instead for 
Did you mean: 

IEEE Floating Point Problems

 
Joseph DiCara
Occasional Visitor

IEEE Floating Point Problems

I have an application writen in C. The application is complied and linked to use IEEE floating point. The enclosed text file is from a debug session. Can any one explain what I am doing wrong.
Honesty is the only policy
4 REPLIES
Stanley F Quayle
Valued Contributor

Re: IEEE Floating Point Problems

It may be late in the middle of the night, but I don't see any problem. Can you be more explicit?

http://www.stanq.com/charon-vax.html
Joseph DiCara
Occasional Visitor

Re: IEEE Floating Point Problems

All three calculations should have the same result. The first answer calculates to 101.355010986328. I then multiply all the numbers (except the divisor by 100) and the result is 10135.5000000000. If the first calculation was correct,then the second answer should have been 10135.5010986328. In the last step. I multiply the divsior by 100 and the result is 101.355000000000. Which is what the first calculation should have returned as the answer.

The question is "Why is the first calculation wrong ?".
Honesty is the only policy
Stanley F Quayle
Valued Contributor

Re: IEEE Floating Point Problems

I'd be willing to bet that if you assigned each of the individual numbers to variables, and looked at their values, one or more won't be represented exactly. You're thinking in terms of decimal, while the computer is binary -- it's sort of the difference between 1/3 and .333333.

You might look at double-precision floating-point, although I'm not sure how to do that in the debugger. That should make the "error" different, but still not zero.

http://www.stanq.com/charon-vax.html
Guy Peleg
Respected Contributor

Re: IEEE Floating Point Problems

I'd like to know what is it that you are trying
to fix. Do you have problems in runtime or
only in the debugger?

From your output it looks like you are using
VMS V7.1, most of the fixes for IEEE related
bugs are not available on V7.1

Guy