Win a copy of Think Java: How to Think Like a Computer Scientist this week in the Java in General forum!

# what accounts for precision problems when computing floating point numbers?

Fendel Coulptz
Greenhorn
Posts: 24
like, 0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1 is not equal to one, but 0.99999999999.

why is this so?

fred rosenberger
lowercase baba
Bartender
Posts: 12122
30
Because computers can't REALLY represent .1

I don't remember exactly how to do it, but to accuratly store it, you'd need an infinite number of places after the decimal point... just like 1/3 is not really 0.3333333333

So, it represents is as close as it can. there are teeny tiny rounding errors, but they can (as you've seen) add up - or NOT add up as the case may be.

hope that helps!!!

Fendel Coulptz
Greenhorn
Posts: 24
i see... so... is there a maximum number of decimal places it can go? like, for eg, anything beyond 0.99999999 gets trucuated, so 0.999999998888245345 will be viewed as 0.99999999

Jeff Bosch
Ranch Hand
Posts: 805
Exactly. Remember, computers do everything in powers of 2, not 10 like us.

fred rosenberger
lowercase baba
Bartender
Posts: 12122
30
you will have this accuracy problem for ALL floating point numbers, whether they are small like 0.000001 or large like 6.024 * 10^23.

the bigger the numbers, the larger the gaps between what can accuratly be represented. if you use this sites search feature, i'm sure you'll find several more threads on this topic. it's a fairly common question... i know i struggled with it when i started programing

Fendel Coulptz
Greenhorn
Posts: 24
thanks fred n jeff