This week's giveaway is in the Android forum. We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line! See this thread for details.

A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi

The reason you don't get exactly the number is the same as why you can't write one-third exactly in decimal. Try it. Get a very sharp pencil and write and don't stop until you have got enough threes to give the exact answer.

No fractional arithmetic can be represented exactly in a 123.45 format unless the factors of the denominator of the fraction are all shared by the radix. So in decimal 10 = 5 X 2, so it is possible to represent such fractions as � � ⅛ 1/5 1/10 1/20 1/25 exactly, not 1/3 or 1/7 or 1/11. In the case of computers, they always use binary arithmetic, so the only fractions which can be accurately represented are those whose denominators have 2 as a factor, nothing else, so � � ⅛ 1/16 1/32 can be represented, not 1/3 1/5 etc. Any other fraction, which includes 0.02 (1/50) can only be approximated. So every now and again you will notice this imprecision. Like 1681.02*12 ->201722.39999999998

There is an explanation on the Javaranch FAQs click here and look for no 20. That gives three links which might provide a better explanation than I did. CR

"We're kind of on the level of crossword puzzle writers... And no one ever goes to them and gives them an award." ~Joe Strummer sscce.org

Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791

posted

0

Thanks, that is easier to digest. I noticed i wrote "Numbers" instead of "Arithmetic" which is the actual title. That was one more work than I could remember for the time it took to type it. Sheesh. This stuff makes me miss COBOL which was designed for currency.

That "heavy" article looks intimidating -- even to a math major. I've had it bookmarked for a long time, with the intention of digging into it some day. But on the other hand, it seems enough just to be aware of the problem (and simply avoiding it) without intimate knowledge of the gory details.

Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 38045

22

posted

0

It is a really heavy article, but it covers all sorts of things other than precision in floating-point arithmetic.

Originally posted by Campbell Ritchie: It is a really heavy article, but it covers all sorts of things other than precision in floating-point arithmetic.

Okay, you talked me into it. I started it last night, and will return to it later today after I've had more coffee.