aspose file tools*
The moose likes JDBC and the fly likes export to XML Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC
Bookmark "export to XML" Watch "export to XML" New topic
Author

export to XML

jonathan mozes
Greenhorn

Joined: Jan 21, 2001
Posts: 13

hey everyone .

i have a problem with jdbc. i have a DB with something like
20,000 records , (and change :-)) , i'm exporting them into
xml file, my problem is that when i retrieve then from the DB
the first 100 is fast , but then it takes me something
like hour to getall the rows, i can open threads but it
still a lot of time, does any one have an idea?
Peter Tran
Bartender

Joined: Jan 02, 2001
Posts: 783
Jonathan,
Have you done any timing to verify that the DB access is your bottleneck? What DB engine are you using? What JDBC driver implementation are you using? I can't believe it's that slow (not that I'm doubting you - which I am. ) From my experience, I would think creating the XML string for the 20K+ records is your bottleneck rather than the DB fetches. What technique are you using to fetch 100 records at a time?
I would recommend commenting out the XML creation process, and see how long it takes to just fetch all those records. For example, you can do this.

-Peter

[This message has been edited by Peter Tran (edited January 21, 2001).]
jonathan mozes
Greenhorn

Joined: Jan 21, 2001
Posts: 13
although i forgot to mention , i'm using JDBC -ODBC bridge , and Oracle7 .
i'll make the test, and get back to you,
lets assume that's not what's keeping me behind. how can i still retrived 20k rows ?
i don't mind doing the XML after i get the rows , but i hv no idea where to keep them..
i can always hold them in a vector , or somthing like that, but i
think it would take me even longer.
thx for the help
Jonathan

Peter Tran
Bartender

Joined: Jan 02, 2001
Posts: 783
Ugh...Using the JDBC-ODBC bridge might be another one of your problem. Read this topic regarding what we think of the JDBC-ODBC bridge. Notice Thomas' comments. If there are memory leaks in the implementation, then it may explain your performance problem. Try running your application with the -verbosegc and see if your memory usage creeps up.
-Peter
jonathan mozes
Greenhorn

Joined: Jan 21, 2001
Posts: 13
recordCount: 5210 - runtime: 441
that's about what it takes me....
somthing like 4 seconds.... :-(
so how can i keep them ? i need to constrcu an xml of them?
any idea?
Jonathan
Peter Tran
Bartender

Joined: Jan 02, 2001
Posts: 783
Jonathan,
Your retrieved 5210 records in less than 1/2 second (441 milliseconds). That's not bad for the JDBC-ODBC bridge. I was expecting it to be much slower. Working the numbers, 20K+ records should only take you 2 seconds to retrieve.
Assuming that the database access is acceptable, then building the XML string will be your bottleneck. How are you doing this now? Don't be bashful, and show us some code my man...
-Peter
PS. Please read the FAQ to learn how to format your code so it's readable.
[This message has been edited by Peter Tran (edited January 21, 2001).]
jonathan mozes
Greenhorn

Joined: Jan 21, 2001
Posts: 13
my code looks like this :
please notice that it works, just very slow...

thx
Jonathan


[This message has been edited by Thomas Paul (edited January 22, 2001).]
Peter Tran
Bartender

Joined: Jan 02, 2001
Posts: 783
Yikes!!!
Horror of horrors...No wonder your program takes so long. Your program is creating a bazillion String objects. You should use a StringBuffer object and use the StringBuffer.append() method to create your XML string. Outside the loop, do StringBuffer.toString() to get a String object. Oh man, the GC must really hate running your application.
Before your while statement where you allocate a new String object, replace that with a StringBuffer buf = new StringBuffer(); Everywhere inside your code where you have "xml = xml +" replace that with buf.append(). Outside, the while loop execute String xmlString = buf.toString(); to get your XML string object.
Make this change and your program will run like the RoadRunner..."beep...beep..."
-Peter
jonathan mozes
Greenhorn

Joined: Jan 21, 2001
Posts: 13
thank :-)
thank :-)
thank :-)
thank :-)
thank :-)
thank :-)
u've just saved me hours of work.. it always happen to me ,
i never knew there's a StringBuffer object, but thanks to you
i know...
i'll give it a try and get back to you with the time it took me.
jonathan

you learn somthing new everyday.

jonathan mozes
Greenhorn

Joined: Jan 21, 2001
Posts: 13

that's how long it took me.... :-)
Time For Fetching Rows : 95737

something like a minute and a half , and i still have to improve it but still................................
thanks a lot .
jonathan mozes
 
wood burning stoves
 
subject: export to XML