File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Beginning Java and the fly likes Strange rounding Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Beginning Java
Bookmark "Strange rounding" Watch "Strange rounding" New topic
Author

Strange rounding

Tom Johnson
Ranch Hand

Joined: May 11, 2005
Posts: 142
Hi,
We have a legacy Math utility class containing the following methods:


I realise it should be using BigDecimal everywhere but for the moment I can't change the code. What we are currently seeing is that in production on Websphere, the call


is returning 100.36999999, rather than 100.37. This does not happen in windows or in Tomcat. I have created a simple script and just called that exact line above (outside the application) and it works fine!

Following through the code, the result of the primitive subtraction is 100.36999999999989, which is seen in windows as well. When this passed to the roundDouble method, the factor is 10^8 = 100000000. This is multiplied by the input so its 100.36999999999989 * 100000000 = 10036999999. This is then rounded to the nearest int which is 10037000000. Finally we divide by the factor so its 10037000000 / 100000000 = 100.37.

I am getting 100.36999999 as the final result...so either the Math.round is truncating 10036999999, rather than rounding it or final divide 10037000000 / 100000000 is going wrong and returning 100.36999999.

Can anyone throw any ideas out why there would be a difference, especially between a running application calling a method and a simple script just calling the same method in the same jar on the same environment but outside the application, or any ideas on what could be going wrong?

Thanks

[ UD: added linebreak to preserve layout ]


<a href="http://faq.javaranch.com/java/UseCodeTags" target="_blank" rel="nofollow">Use Code Tags!!</a>
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 41466
    
  51
Not sure what's going on, but you may want to look into using java.lang.StrictMath instead of java.lang.Math. Its API is identical, but it produces identical results across platforms (which Math is not required to do).


Ping & DNS - my free Android networking tools app
Tom Johnson
Ranch Hand

Joined: May 11, 2005
Posts: 142
Thanks for the pointer....although I see that Math.pow simply delegates to StrictMath.pow while Math.round and StrictMath.round are the implemented exactly the same:


where floor is Math class simply delegates to StrictMath.floor.

So shouldn't calls to Math/StrictMath pow and round always return the same result, cross platform as well?
Ulf Dittmer
Marshal

Joined: Mar 22, 2005
Posts: 41466
    
  51
I've never checked what StrictMath does exactly. From what you say it sounds as if both should do the same thing in this case. Which means that any platform differences must originate elsewhere in the calculation. I'd do a step-by-step comparison of the intermediate values on both platforms to determine where that happens.
Tom Johnson
Ranch Hand

Joined: May 11, 2005
Posts: 142
Yep I tried that....the code that seems to be going wrong is in a utility jar that we use, which I cannot change. So to do what you suggest, I copied the methods from the utility jar into my class and called them instead....should be the same thing. Unfortunately not! I couldn't get the error to happen calling the copied code, even though its identical! I can only think the utility jar is compiled "differently" than my copied version....or does this even even make sense?
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 38334
    
  23
If I remember correctly, the simple Math class methods (eg max, min, abs) use their own code,something like thisand the "complicated" methods simply call StrictMath, like thisAt least I think most of them do that.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Strange rounding