aspose file tools*
The moose likes JDBC and the fly likes double data type and output Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC
Bookmark "double data type and output" Watch "double data type and output" New topic
Author

double data type and output

Howard Watson
Ranch Hand

Joined: Jan 07, 2004
Posts: 56
Pervasive SQL 2000i sp4, jdbc.v2 type4 driver

The output below is from a java servlet. The same query in the Pervasive SQL Data Manager does not result in any calculated results with 13 digits to the right of the decimal point. None of the output is formatted numbers just whatever is stored in the variable.

Any ideas where this is coming from. I'd like to understand or resolve this before I apply formatting.

Query: SELECT unitsAlloc,priceUnit,(priceUnit * unitsAlloc) AS dollarsUsed FROM table

priceUnit and unitsAlloc are Pervasive SQL double data types

String sUnitsAlloc = String.valueOf(rs.getDouble("unitsAlloc"));
String sPriceUnit = String.valueOf(rs.getDouble("priceUnit"));
String sDollarsUsed = String.valueOf(rs.getDouble("dollarsUsed"));

//also tried this to calculate dollarsUsed
double dUnitsAlloc = rs.getDouble("unitsAlloc");
double dPriceUnit = rs.getDouble("priceUnit");
double dDollarsUsed = dUnitsAlloc * dPriceUnit;
String sDollarsUsed = String.valueOf(dDollarsUsed);

//output
while(rs.next()) {
out.println("<td>" + sUnitsAlloc + "</td>");
out.println("<td>" + sPriceUnit + "</td>");
out.println("<td>" + sDollarsUsed + "</td>");
}

Some sample output:

Units Used Unit Price Dollars Used
1.0 150.0 150.0
1.0 150.0 150.0
1.0 150.0 150.0
1.0 150.0 150.0
1.0 150.0 150.0
1.0 150.0 150.0
13.0 62.83 816.79
22.0 62.83 1382.26
20.0 62.83 1256.6
22.0 62.83 1382.26
21.0 62.83 1319.43
21.0 62.83 1319.43
21.0 62.83 1319.43
10.0 84.07 840.6999999999999
20.0 84.07 1681.3999999999999
23.0 84.07 1933.61
20.0 84.07 1681.3999999999999
20.0 84.07 1681.3999999999999
22.0 84.07 1849.54
20.0 84.07 1681.3999999999999
22.0 84.07 1849.54
19.0 84.07 1597.33
0.0 2.84 0.0
0.0 2.84 0.0
30.25 2.84 85.91
116.0 2.84 329.44
0.0 2.84 0.0
105.0 2.84 298.2
105.0 2.84 298.2
77.0 2.84 218.67999999999998
113.0 2.84 320.91999999999996
1.0 150.0 150.0
1.0 150.0 150.0
1.0 150.0 150.0
18.0 58.71 1056.78
22.0 58.71 1291.6200000000001
20.0 58.71 1174.2
31.0 73.13 2267.0299999999997
31.0 73.13 2267.0299999999997
30.0 73.13 2193.8999999999996
14.0 35.02 490.28000000000003
Jeanne Boyarsky
author & internet detective
Marshal

Joined: May 26, 2003
Posts: 30919
    
158

Howard,
If not sure I understand what you are expecting as output. Are you saying that the Data Manager gives you zeros? Or is this about the precision of the doubles? Or something else entirely?


[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
Howard Watson
Ranch Hand

Joined: Jan 07, 2004
Posts: 56
Hi Jeanne,

Did a little research while writing this. Remember why I liked BCD.
I can let this go. Thanks.

I expected:
22.0 * 58.71 = 1291.62 and not 1291.6200000000001
113.0 * 2.84 = 320.92 and not 320.91999999999996
30.0 * 73.13 = 2193.90 and not 2193.8999999999996...

Maybe it is something I don't remember about floating point numeric processing. I don't see a pattern in the results that explains why
18.0 * 58.71 = 1056.78 and
22.0 * 58.71 = 1291.6200000000001

The Data Manager (a query tool) returns 22.0 * 58.71 = 1291.62, but maybe it is rounding internally.
Jeanne Boyarsky
author & internet detective
Marshal

Joined: May 26, 2003
Posts: 30919
    
158

Howard,
One of our sherriffs wrote an article on this last year.
Adeel Ansari
Ranch Hand

Joined: Aug 15, 2004
Posts: 2874
wiered. i haven't got into this kinda problem.
Jeanne i love you for this.

thanks
[ November 21, 2004: Message edited by: adeel ansari ]
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: double data type and output