Win a copy of Zero to AI - A non-technical, hype-free guide to prospering in the AI era this week in the Artificial Intelligence and Machine Learning forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Paul Clapham
  • Bear Bibeault
  • Jeanne Boyarsky
Sheriffs:
  • Ron McLeod
  • Tim Cooke
  • Devaka Cooray
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Jj Roberts
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • salvin francis
  • Scott Selikoff
  • fred rosenberger

What is the correct result of this computation?

 
Bartender
Posts: 1757
17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Using BigDecimal for the simple calculation below, I get one result in Java (and Python). But using an online calculator and another program, I get another result.

Java Code:



Result:

0.000000000000045454545454545453631211255875255243966038341187705575727952898226001045920632102272727272727272727272727272727272727272727272727272727273

----

But, at this website: https://apfloat.appspot.com/

I get a different result:

.000000000001 / 22
0.00000000000004545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545454545

--

Which is correct and why?

Thanks,

-- mike
 
Master Rancher
Posts: 3692
44
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
The second is correct - at least, it's what most of us would want.  To see what's wrong with the first, add this line:

I get

which is not what we expect for something that should be 0.000000000001.  Unfortunately, that's the sort of thing that can happen when we use the double primitive type.  The solution is to not use the constructor BigDecimal(double) - see the [url=https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/math/BigDecimal.html#%3Cinit%3E(double)]API for BigDecimal(double) for an explanation why.  Instead, try using BigDecimal(String):

That should give you what you want...
 
Mike London
Bartender
Posts: 1757
17
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Mike Simmons wrote:The second is correct - at least, it's what most of us would want.  To see what's wrong with the first, add this line:

I get

which is not what we expect for something that should be 0.000000000001.  Unfortunately, that's the sort of thing that can happen when we use the double primitive type.  The solution is to not use the constructor BigDecimal(double) - see the [url=https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/math/BigDecimal.html#%3Cinit%3E(double)]API for BigDecimal(double) for an explanation why.  Instead, try using BigDecimal(String):

That should give you what you want...



Thanks very much!

- mike
 
Marshal
Posts: 70625
288
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Also look up the BigDecimal#valueOf(double) method and see whether that would help you.
 
Ranch Hand
Posts: 574
VI Editor Chrome Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You could also do 1/22, the only thing that changes is the decimal point.  This shows the second result is indeed correct (adjusting the decimal point, of course).

As others have said, you've just discovered the joys of big numbers and floating point math on computers.

 
Campbell Ritchie
Marshal
Posts: 70625
288
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Jim Venolia wrote:You could also do 1/22 . . .

You mean 1.0 / 22; 1 / 22 evaluates to 0 
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic