| Author |
Strange value returned
|
michael yue
Ranch Hand
Joined: Nov 20, 2003
Posts: 204
|
|
Hi I need some help here. I am trying to display some calculations in a manner of sum of money. Example 120.00, 234.23 and so on. I discovered something weird when I did this double value=0.01*45878; Instead of returning 458.78 it returns 458.78000000000003 The other numbers does not get affected. What is wrong? Are there any way I could display to .00 precision for this particular strange value. Thanks a lot
|
 |
Chandra Sekhar
Ranch Hand
Joined: Sep 26, 2003
Posts: 90
|
|
Plz check the link another one i'm confused with the explanation given.Can some one explain this little more Thanks in Advance Chandrasekhar S SCJP [ June 18, 2004: Message edited by: Chandrasekhar S ]
|
"Luck is when opportunity meets preparation"
|
 |
Darin Niard
Ranch Hand
Joined: Jun 08, 2004
Posts: 118
|
|
Check out the round method in the Math class. Hint: Temporarily move the decimal over two spots while you round.
|
 |
Eddie Vanda
Ranch Hand
Joined: Mar 18, 2003
Posts: 281
|
|
The problem is that a float and double are each stored as two parts, a mantissa and an exponent and that they are each in terms of powers of two. It's the exponent that gives floating point such a wide range, but at the expense of mantissa accuracy. I think that mantissa float accuracy is about 24 binary bits and about 48 bits for double. If you divide those by 3 (a very rough guide), you get about 8 and 16 decimal digit accuracy. Iow, you should not depend on more than about 6 or 14 bit accuracy respectively for each one. Use the format classes to produce no more than the required number of digits. [ June 19, 2004: Message edited by: Eddie Vanda ]
|
The nice thing about Standards is that there are so many to choose from!
|
 |
 |
|
|
subject: Strange value returned
|
|
|