# Decimal place result

henry akoma

Greenhorn

Posts: 21

posted 10 years ago

- 0

hi guys,

some colleagues and i were writting a calculator program, but the result of some of our calculations were giving approximate answers and not the exact one.for example:

subtraction gave -0.100000024 instead of -0.1.

please can anyone help us with what to do to get the correct answer and may be decimal place

some colleagues and i were writting a calculator program, but the result of some of our calculations were giving approximate answers and not the exact one.for example:

subtraction gave -0.100000024 instead of -0.1.

please can anyone help us with what to do to get the correct answer and may be decimal place

Ulf Dittmer

Rancher

Posts: 42966

73

posted 10 years ago

- 0

floats are inherently limited in their precision, and some numbers just can't be expressed precisely using them. "0.1" is such a number, as is "0.9". Using doubles makes things a bit better (around 14 exact decimals instead of 7 or so), but the basic problem is the same. (If you're interested in the gory details, read up on What Every Computer Scientist Should Know About Floating-Point Arithmetic.)

If you require exact floating point arithmetic, you can use the java.math.BigDecimal class.

[ August 09, 2005: Message edited by: Ulf Dittmer ]

If you require exact floating point arithmetic, you can use the java.math.BigDecimal class.

[ August 09, 2005: Message edited by: Ulf Dittmer ]

Stuart Gray

Ranch Hand

Posts: 410

posted 10 years ago

- 0

There are well known problems with floating point arithmetic. You could try BigDecimal in the java.math package.

With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime. |