aspose file tools*
The moose likes Beginning Java and the fly likes Precision? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Beginning Java
Bookmark "Precision?" Watch "Precision?" New topic
Author

Precision?

Mark Howard
Ranch Hand

Joined: Feb 14, 2001
Posts: 285
This program:

... produces this output: "Total = 99.99905"
Not 100 as expected. I believe this loss of precision is a well-documented Java fact.
All I want to know is ... WHY?
Junilu Lacar
Bartender

Joined: Feb 26, 2001
Posts: 4462
    
    6

I believe it has to do a lot with the limitations of representing decimal values in binary. If precision or loss thereof is a concern, maybe you could try using the java.lang.StrictMath class.
HTH
J. Lacar

... produces this output: "Total = 99.99905"
Not 100 as expected. I believe this loss of precision is a well-documented Java fact.
All I want to know is ... WHY? [/B]


[This message has been edited by JUNILU LACAR (edited February 28, 2001).]


Junilu - [How to Ask Questions] [How to Answer Questions]
Cindy Glass
"The Hood"
Sheriff

Joined: Sep 29, 2000
Posts: 8521
The loss of precision in both floats and doubles is a well documented PROGRAMMING fact. It impacts every language known to man. The issue is that a TRUE float or double would have an infinite number of decimal places. Obviously a computer can not do that. Therefore some rounding concession must be made. Each language handles the rounding issue differently, but for all of them you can only say the float or double is "ABOUT this value".


"JavaRanch, where the deer and the Certified play" - David O'Meara
Mark Howard
Ranch Hand

Joined: Feb 14, 2001
Posts: 285
Thanks for the response.
This loss of precision explains my depleting bank balance
[This message has been edited by mark howard (edited February 28, 2001).]
Cindy Glass
"The Hood"
Sheriff

Joined: Sep 29, 2000
Posts: 8521
Yeah, I also have a float value in my account. I keep trying to cast it into a double but. . . . .
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Precision?