Help coderanch get a
new server
by contributing to the fundraiser
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

What is the correct result of this computation?

 
Bartender
Posts: 1971
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Using BigDecimal for the simple calculation below, I get one result in Java (and Python). But using an online calculator and another program, I get another result.

Java Code:



Result:

0.000000000000045454545454545453631211255875255243966038341187705575727952898226001045920632102272727272727272727272727272727272727272727272727272727273

----

But, at this website: https://apfloat.appspot.com/

I get a different result:

.000000000001 / 22
0.00000000000004545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545

--

Which is correct and why?

Thanks,

-- mike
 
Master Rancher
Posts: 4932
75
  • Likes 1
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The second is correct - at least, it's what most of us would want.  To see what's wrong with the first, add this line:

I get

which is not what we expect for something that should be 0.000000000001.  Unfortunately, that's the sort of thing that can happen when we use the double primitive type.  The solution is to not use the constructor BigDecimal(double) - see the [url=https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/math/BigDecimal.html#%3Cinit%3E(double)]API for BigDecimal(double) for an explanation why.  Instead, try using BigDecimal(String):

That should give you what you want...
 
Mike London
Bartender
Posts: 1971
17
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Mike Simmons wrote:The second is correct - at least, it's what most of us would want.  To see what's wrong with the first, add this line:

I get

which is not what we expect for something that should be 0.000000000001.  Unfortunately, that's the sort of thing that can happen when we use the double primitive type.  The solution is to not use the constructor BigDecimal(double) - see the [url=https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/math/BigDecimal.html#%3Cinit%3E(double)]API for BigDecimal(double) for an explanation why.  Instead, try using BigDecimal(String):

That should give you what you want...



Thanks very much!

- mike
 
Marshal
Posts: 79522
379
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Also look up the BigDecimal#valueOf(double) method and see whether that would help you.
 
Ranch Hand
Posts: 574
VI Editor Chrome Linux
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
You could also do 1/22, the only thing that changes is the decimal point.  This shows the second result is indeed correct (adjusting the decimal point, of course).

As others have said, you've just discovered the joys of big numbers and floating point math on computers.

 
Campbell Ritchie
Marshal
Posts: 79522
379
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jim Venolia wrote:You could also do 1/22 . . .

You mean 1.0 / 22; 1 / 22 evaluates to 0 
 
She'll be back. I'm just gonna wait here. With this tiny ad:
We need your help - Coderanch server fundraiser
https://coderanch.com/t/782867/Coderanch-server-fundraiser
reply
    Bookmark Topic Watch Topic
  • New Topic