# problem/consult with precision of division with BigDecimals

Manuel Jordan

Ranch Hand

Posts: 131

posted 8 years ago

- 0

Hello guys

i have this situation

a and b 2 BigDecimals

a=14.00

b=6.00

for a simple calculator

a/b = 2.3333......

so for humans 14/6 = 2.33 good by infinite

for java 14/6 = 2.34 (a.divide(b,2))

but with (a.divide(b,3)) 2.33

why?

it should be the same?

regards

i have this situation

a and b 2 BigDecimals

a=14.00

b=6.00

for a simple calculator

a/b = 2.3333......

so for humans 14/6 = 2.33 good by infinite

for java 14/6 = 2.34 (a.divide(b,2))

but with (a.divide(b,3)) 2.33

why?

it should be the same?

regards

kill your pride, share your knowledge with all

Jim Yingst

Wanderer

Sheriff

Sheriff

Posts: 18671

posted 8 years ago

- 0

The reason for the difference between 2.34 and 2.33 is that when you say a.divide(b,2)), that "2" is the rounding mode - which is BigDecimal.ROUND_CEILING. While "3" is BigDecimal.ROUND_FLOOR. In one case you're telling the computer to round up, and in the other you're telling it to round down. No surprise there. If you want it to round to whichever is

If you're asking why the result is 2.33 rather than 2.33333 or 2.3333333333333 or 2.333333333333333333333333333333333 or whatever, then the answer is that you need to tell it how many digits you want. If you want 10 digits after the decimal point, use a scale of 10:

a.divide(b,10, BigDecimal.ROUND_CEIL) = 2.3333333334

a.divide(b,10, BigDecimal.ROUND_FLOOR) = 2.3333333333

If you want more digits or less, use another number besides 10.

Since you originally did

*closest*, use BigDecimal.ROUND_HALF_DOWN or BigDecimal.ROUND_HALF_UP. The difference is very minor. You can examine the JavaDoc to learn the exact differences between rounding modes.If you're asking why the result is 2.33 rather than 2.33333 or 2.3333333333333 or 2.333333333333333333333333333333333 or whatever, then the answer is that you need to tell it how many digits you want. If you want 10 digits after the decimal point, use a scale of 10:

a.divide(b,10, BigDecimal.ROUND_CEIL) = 2.3333333334

a.divide(b,10, BigDecimal.ROUND_FLOOR) = 2.3333333333

If you want more digits or less, use another number besides 10.

Since you originally did

*not*specify the number of digits you wanted, BigDecimal assumed you wanted the same number as were used for the first BigDecimal in the calculation, a. Since you created this as 14.00, it had two digits after the decimal. That's what was used in your original answers, 2.34 and 2.33."I'm not back." - Bill Harding, *Twister*

I agree. Here's the link: http://aspose.com/file-tools |