# Math Precision - Calculator

Bruce Martin

Greenhorn

Posts: 3

posted 13 years ago

Hi all,

I am new to the forums, but am hoping someone can help me out.

I am trying to figure some things out:

Using M$ Windows calculator - if you enter 5 sqrt * = you get 5 - same as sqrt(5) * sqrt(5)

In java i have tried:

BigDecimal bd1 = new BigDecimal(Math.sqrt(5)).setScale(32, BigDecimal.ROUND_HALF_UP);

BigDecimal bd2 = bd1;

System.out.println(bd1.multiply(bd1).toString());

This outputs: 5.0000000000000004858631542817679579261519802454185847561882183321

instead of 5

I have also tried:

System.out.println(Math.sqrt(5) * Math.sqrt(5));

this outputs: 5.000000000000001

Also using M$ calculator - 1 / 3 * 3 returns 1.

I tried this with BigDecimal:

bd1 = new BigDecimal("3").setScale(32, BigDecimal.ROUND_HALF_UP);

bd2 = new BigDecimal("1").setScale(32, BigDecimal.ROUND_HALF_UP);

System.out.println(bd2.divide(bd1, BigDecimal.ROUND_HALF_UP).multiply(bd1).toString());

and get: 0.9999999999999999999999999999999900000000000000000000000000000000

Maybe I need to refactor this equation to be (1 * 3) / 3 - this would return 1.

I am trying to build a calculator, but I can't seem to work through some of these issues.

Any help is greatly appreciated.

I am new to the forums, but am hoping someone can help me out.

I am trying to figure some things out:

Using M$ Windows calculator - if you enter 5 sqrt * = you get 5 - same as sqrt(5) * sqrt(5)

In java i have tried:

BigDecimal bd1 = new BigDecimal(Math.sqrt(5)).setScale(32, BigDecimal.ROUND_HALF_UP);

BigDecimal bd2 = bd1;

System.out.println(bd1.multiply(bd1).toString());

This outputs: 5.0000000000000004858631542817679579261519802454185847561882183321

instead of 5

I have also tried:

System.out.println(Math.sqrt(5) * Math.sqrt(5));

this outputs: 5.000000000000001

Also using M$ calculator - 1 / 3 * 3 returns 1.

I tried this with BigDecimal:

bd1 = new BigDecimal("3").setScale(32, BigDecimal.ROUND_HALF_UP);

bd2 = new BigDecimal("1").setScale(32, BigDecimal.ROUND_HALF_UP);

System.out.println(bd2.divide(bd1, BigDecimal.ROUND_HALF_UP).multiply(bd1).toString());

and get: 0.9999999999999999999999999999999900000000000000000000000000000000

Maybe I need to refactor this equation to be (1 * 3) / 3 - this would return 1.

I am trying to build a calculator, but I can't seem to work through some of these issues.

Any help is greatly appreciated.

zoe goddard

Ranch Hand

Posts: 74

posted 13 years ago

I'm not sure what kind of answer you're looking for. I'm assuming you're trying to get rid of all those extra decimal places. You could try to using NumberFormat instead of the toString()

Bruce Martin

Greenhorn

Posts: 3

posted 13 years ago

I think the calculator is factoring out unnecessary (or negating) operations, but I wonder how. I am trying to figure out how I would do something like that in java. The more I think about it, the more complex it gets.

two even more complex examples (in M$ Calc - standard view):

(these are the key sequences)

5 sqrt + 5 sqrt + 5 sqrt = * = returns 45 - is the calculator actually doing algebra here??

5 sqrt 1/x * 2 = 1/x * 2 = * = returns 5

Does anyone have any suggestions on how one could factor out operations that cancel each other out in a sequence of calculator operations?

Originally posted by zoe goddard:

I'm not sure what kind of answer you're looking for. I'm assuming you're trying to get rid of all those extra decimal places. You could try to using NumberFormat instead of the toString()

I think the calculator is factoring out unnecessary (or negating) operations, but I wonder how. I am trying to figure out how I would do something like that in java. The more I think about it, the more complex it gets.

two even more complex examples (in M$ Calc - standard view):

(these are the key sequences)

5 sqrt + 5 sqrt + 5 sqrt = * = returns 45 - is the calculator actually doing algebra here??

5 sqrt 1/x * 2 = 1/x * 2 = * = returns 5

Does anyone have any suggestions on how one could factor out operations that cancel each other out in a sequence of calculator operations?

David Weitzman

Ranch Hand

Posts: 1365

posted 13 years ago

The calculator isn't doing rocket science or genetic splicing or something cool like that. It probably ended up with an answer much like the one you got above by simply doing exactly what you told it. The difference between your answer and its -- it rounds everything for display on the screen.

But if you do want to factor out operations that cancel each other...

Build a tree of operations and numbers and make a visitor that recognizes optimizable sections (pairs of inverse functions or other weird patterns). Optimization can be done at operation insertion time or just before execute()ing a node. If you aren't familiar with this sort of tree (I suppose it doesn't really qualify as an Abstract Syntax Tree unless it came from parsing something) then I can't think of a good place to point you. This is probably overkill though.

But if you do want to factor out operations that cancel each other...

Build a tree of operations and numbers and make a visitor that recognizes optimizable sections (pairs of inverse functions or other weird patterns). Optimization can be done at operation insertion time or just before execute()ing a node. If you aren't familiar with this sort of tree (I suppose it doesn't really qualify as an Abstract Syntax Tree unless it came from parsing something) then I can't think of a good place to point you. This is probably overkill though.

Bruce Martin

Greenhorn

Posts: 3

posted 13 years ago

But I don't think it just rounds off for display purposes, because if you copy the result of 5 sqrt to the clipboard, clear the calculator and paste it back into the calculator and multiply it by istself you get:

4.9999999999999999999999999999988

whereas 5 sqrt * 5 sqrt returns 5.

So if the calculator were simply rounding everything for display on the screen, wouldn't it do it in all situations?

If it is simply rounding the answer, how does it know when to round and when not to round?

[ June 20, 2002: Message edited by: Bruce Martin ]

Originally posted by David Weitzman:

The calculator isn't doing rocket science or genetic splicing or something cool like that. It probably ended up with an answer much like the one you got above by simply doing exactly what you told it. The difference between your answer and its -- it rounds everything for display on the screen.

But I don't think it just rounds off for display purposes, because if you copy the result of 5 sqrt to the clipboard, clear the calculator and paste it back into the calculator and multiply it by istself you get:

4.9999999999999999999999999999988

whereas 5 sqrt * 5 sqrt returns 5.

So if the calculator were simply rounding everything for display on the screen, wouldn't it do it in all situations?

If it is simply rounding the answer, how does it know when to round and when not to round?

[ June 20, 2002: Message edited by: Bruce Martin ]

David Weitzman

Ranch Hand

Posts: 1365

posted 13 years ago

Maybe it does something more interesting then I'd guessed. Two possibilities are that when you copy and paste it lost a few useful bits at the end (that couldn't fit on the display anyway) or that it chooses where to round based on several factors (like how many digits you entered and what operations you applied). All things considered, there's more than one way to write a calculator. The issue with push button calculators is that the user needs to see an intermediate result before the calculation is done. Perhaps a stack would work better than a tree.