Hi,
We have a legacy Math utility class containing the following methods:
I realise it should be using BigDecimal everywhere but for the moment I can't change the code. What we are currently seeing is that in production on Websphere, the call
is returning 100.36999999, rather than 100.37. This does not happen in windows or in Tomcat. I have created a simple script and just called that exact line above (outside the application) and it works fine!
Following through the code, the result of the primitive subtraction is 100.36999999999989, which is seen in windows as well. When this passed to the roundDouble method, the factor is 10^8 = 100000000. This is multiplied by the input so its 100.36999999999989 * 100000000 = 10036999999. This is then rounded to the nearest int which is 10037000000. Finally we divide by the factor so its 10037000000 / 100000000 = 100.37.
I am getting 100.36999999 as the final result...so either the Math.round is truncating 10036999999, rather than rounding it or final divide 10037000000 / 100000000 is going wrong and returning 100.36999999.
Can anyone throw any ideas out why there would be a difference, especially between a running application calling a method and a simple script just calling the same method in the same jar on the same environment but outside the application, or any ideas on what could be going wrong?
Thanks
[ UD: added linebreak to preserve layout ]