File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Java in General and the fly likes Precision for double literals Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of OCM Java EE 6 Enterprise Architect Exam Guide this week in the OCMJEA forum!
JavaRanch » Java Forums » Java » Java in General
Bookmark "Precision for double literals" Watch "Precision for double literals" New topic
Author

Precision for double literals

Lee Bulmer
Greenhorn

Joined: Apr 25, 2009
Posts: 2
Fairly trivial question...



Running this gives the output:

d = 0.8999999999999999
d2 = 0.9

What's bugging me is the second line in the output. It's not possible to store the value 0.9 precisely in the double variable, hence the first line of the output. That's fine. I get that. But, that being the case, I don't really understand why the second line doesn't print "d2 = 0.8999999999999999". How is the value 0.9 being held precisely between lines 2 and 4. I'm just curious about what is happening under-the-hood and I can't seem to find the relevant part of the JLS to explain it.

Thanks in advance.
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18765
    
  40

I'm just curious about what is happening under-the-hood and I can't seem to find the relevant part of the JLS to explain it.


The floating point formats (and behavior) is not defined in the JLS. Java uses the IEEE standard for floating points -- 32 bit and 64 bit standard.

Henry


Books: Java Threads, 3rd Edition, Jini in a Nutshell, and Java Gems (contributor)
Lee Bulmer
Greenhorn

Joined: Apr 25, 2009
Posts: 2
Henry Wong wrote:
I'm just curious about what is happening under-the-hood and I can't seem to find the relevant part of the JLS to explain it.


The floating point formats (and behavior) is not defined in the JLS. Java uses the IEEE standard for floating points -- 32 bit and 64 bit standard.

Henry


OK, I see that in the JLS. But, as I understand it*, it specifies how floating point numbers are encoded into binary. Using this encoding it's not possible to precisely specify the number 0.9 (for example). This causes the rounding error in the assignment to variable d in the example I posted. This same rounding error does not occur in the assignment to variable d2. This leads to to believe that the JVM is not using IEEE 754 to encode the literal 0.9 in the assignment to d2.

If this behaviour is explained somewhere in IEEE 754, I've missed it. I also cannot see anything in the JLS that suggests the JVM ever uses anything other than IEEE 754 to encode floating point numbers. Obviously I've missed something in one of the two specifications. So, if you happen to know a part of IEEE 754 spec that would explains this, I would be very grateful if you could point it out as it's esaping me for the moment.

Thanks for the reply,

Lee.
* I admit my understanding may not be perfect, hence the post.
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 38519
    
  23
I have never actually read IEEE754(1985) which I think is now called something different [but I do know what the structure of floating-point numbers is]. But it was unnecessary to explain imprecision; programmers in those days had used slide rules (I still have an old slide rule I bought about 1968) and were familiar with imprecision. Look in our FAQ (no 20) where it gives some useful links.
Steve Luke
Bartender

Joined: Jan 28, 2003
Posts: 4181
    
  21

It also might be worth noting that with the simple example given above, the compiler can do some optimizations because it knows the value of d is calculated from two constants when assigned. Which means the results is a constant. Which means the math can be done at compile time rather than run time. On the other hand, the value of d and d2 are never changed, so they can be replaced by constants as well. Given this code:

You can get this byte code (un-important stuff cleaned out)


So when d is encountered it is treated like the constant 0.89999...d, while when d2 is encountered it is treated as the constant 0.9d. Not sure if it helps, or is exactly pertinent to your question, but it is something to keep in mind when making these sorts of comparisons.


Steve
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Precision for double literals