aspose file tools*
The moose likes Beginning Java and the fly likes what accounts for precision problems when computing floating point numbers? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Java » Beginning Java
Bookmark "what accounts for precision problems when computing floating point numbers?" Watch "what accounts for precision problems when computing floating point numbers?" New topic
Author

what accounts for precision problems when computing floating point numbers?

Fendel Coulptz
Greenhorn

Joined: Sep 23, 2004
Posts: 24
like, 0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1 is not equal to one, but 0.99999999999.

why is this so?
fred rosenberger
lowercase baba
Bartender

Joined: Oct 02, 2003
Posts: 11402
    
  16

Because computers can't REALLY represent .1

I don't remember exactly how to do it, but to accuratly store it, you'd need an infinite number of places after the decimal point... just like 1/3 is not really 0.3333333333

So, it represents is as close as it can. there are teeny tiny rounding errors, but they can (as you've seen) add up - or NOT add up as the case may be.

hope that helps!!!


There are only two hard things in computer science: cache invalidation, naming things, and off-by-one errors
Fendel Coulptz
Greenhorn

Joined: Sep 23, 2004
Posts: 24
i see... so... is there a maximum number of decimal places it can go? like, for eg, anything beyond 0.99999999 gets trucuated, so 0.999999998888245345 will be viewed as 0.99999999
Jeff Bosch
Ranch Hand

Joined: Jul 30, 2003
Posts: 805
Exactly. Remember, computers do everything in powers of 2, not 10 like us.


Give a man a fish, he'll eat for one day. Teach a man to fish, he'll drink all your beer.
Cheers, Jeff (SCJP 1.4, SCJD in progress, if you can call that progress...)
fred rosenberger
lowercase baba
Bartender

Joined: Oct 02, 2003
Posts: 11402
    
  16

you will have this accuracy problem for ALL floating point numbers, whether they are small like 0.000001 or large like 6.024 * 10^23.

the bigger the numbers, the larger the gaps between what can accuratly be represented. if you use this sites search feature, i'm sure you'll find several more threads on this topic. it's a fairly common question... i know i struggled with it when i started programing
Fendel Coulptz
Greenhorn

Joined: Sep 23, 2004
Posts: 24
thanks fred n jeff
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: what accounts for precision problems when computing floating point numbers?