File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Beginning Java and the fly likes Strange rounding Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Java » Beginning Java
Bookmark "Strange rounding" Watch "Strange rounding" New topic
Author

Strange rounding

Tom Johnson
Ranch Hand

Joined: May 11, 2005
Posts: 142
Hi,
We have a legacy Math utility class containing the following methods:


I realise it should be using BigDecimal everywhere but for the moment I can't change the code. What we are currently seeing is that in production on Websphere, the call


is returning 100.36999999, rather than 100.37. This does not happen in windows or in Tomcat. I have created a simple script and just called that exact line above (outside the application) and it works fine!

Following through the code, the result of the primitive subtraction is 100.36999999999989, which is seen in windows as well. When this passed to the roundDouble method, the factor is 10^8 = 100000000. This is multiplied by the input so its 100.36999999999989 * 100000000 = 10036999999. This is then rounded to the nearest int which is 10037000000. Finally we divide by the factor so its 10037000000 / 100000000 = 100.37.

I am getting 100.36999999 as the final result...so either the Math.round is truncating 10036999999, rather than rounding it or final divide 10037000000 / 100000000 is going wrong and returning 100.36999999.

Can anyone throw any ideas out why there would be a difference, especially between a running application calling a method and a simple script just calling the same method in the same jar on the same environment but outside the application, or any ideas on what could be going wrong?

Thanks

[ UD: added linebreak to preserve layout ]


<a href="http://faq.javaranch.com/java/UseCodeTags" target="_blank" rel="nofollow">Use Code Tags!!</a>
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 39535
    
  27
Not sure what's going on, but you may want to look into using java.lang.StrictMath instead of java.lang.Math. Its API is identical, but it produces identical results across platforms (which Math is not required to do).


Ping & DNS - updated with new look and Ping home screen widget
Tom Johnson
Ranch Hand

Joined: May 11, 2005
Posts: 142
Thanks for the pointer....although I see that Math.pow simply delegates to StrictMath.pow while Math.round and StrictMath.round are the implemented exactly the same:


where floor is Math class simply delegates to StrictMath.floor.

So shouldn't calls to Math/StrictMath pow and round always return the same result, cross platform as well?
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 39535
    
  27
I've never checked what StrictMath does exactly. From what you say it sounds as if both should do the same thing in this case. Which means that any platform differences must originate elsewhere in the calculation. I'd do a step-by-step comparison of the intermediate values on both platforms to determine where that happens.
Tom Johnson
Ranch Hand

Joined: May 11, 2005
Posts: 142
Yep I tried that....the code that seems to be going wrong is in a utility jar that we use, which I cannot change. So to do what you suggest, I copied the methods from the utility jar into my class and called them instead....should be the same thing. Unfortunately not! I couldn't get the error to happen calling the copied code, even though its identical! I can only think the utility jar is compiled "differently" than my copied version....or does this even even make sense?
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 36453
    
  15
If I remember correctly, the simple Math class methods (eg max, min, abs) use their own code,something like thisand the "complicated" methods simply call StrictMath, like thisAt least I think most of them do that.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Strange rounding
 
Similar Threads
rounding a double value
How to round last 2 digits of a double
element of the array from another class
Generating compound interest with integers
Rounding BigDecimal Calculation Yielded Wrong Result