• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Losing precision converting Number to Float

 
Ranch Hand
Posts: 241
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
The following snippet

provides the output
666666.77
666666.75

Why did it round the float convertion since "666666.77" is less than Float.MAX_VALUE ?

Regards,
 
Rancher
Posts: 43081
77
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
See #20 in the JavaBeginnersFaq.
 
Java Cowboy
Posts: 16084
88
Android Scala IntelliJ IDE Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Floating-point data types such as float and double are not infinitely precise.

That means that in a float you cannot just store any number between the min. and max. possible values with any arbitrary number of decimal digits.

The precision of a float is about 6 or 7 decimal digits, so if you try to store numbers with more digits (such as 666666.77 which is 8 digits) then you'll see rounding errors.

In addition to that float and double store numbers as binary fractions instead of decimal fractions. Not all values that in the decimal number system can be represented by a finite number of digits can be stored exactly in the binary system (for example the value 0.1). See Ulf's link for details.
[ March 13, 2007: Message edited by: Jesper Young ]
 
Politics n. Poly "many" + ticks "blood sucking insects". Tiny ad:
a bit of art, as a gift, the permaculture playing cards
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic