File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
The moose likes XML and Related Technologies and the fly likes Tackling large size XML files Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "Tackling large size XML files" Watch "Tackling large size XML files" New topic

Tackling large size XML files

Anil C

Joined: Jan 08, 2001
Posts: 16
I have a requirement to port large amount of data from one DB to another via XML. We are estimating the size of the XML file (my XML includes data from more than a score tables) in the filesystem to be somewhere near 20MB. In the process of uploading, if I have to load this XML file to memory as a DOM object, then it will take up almost 40MB. (or more ?). Is there any way to avoid loading the whole XML but extract only portions ( say, table by table...Due to some technical limitations, we found the option of having separate XML files for each table not viable.)and then extract data phase by phase ? SAX and jDOM have been suggested as possible solutions...I am new to both and as there is a time constraint, will appreciate if anyone can throw more light on the best approach possible
Manuel Palacio
Ranch Hand

Joined: Oct 16, 2000
Posts: 45
I was faced with a similar problem-- a rdf file containing thousands of objects and I solved it with SAX. Use SAX to parse the file. This way you avoid creating the whole tree in memory. And to create a large document use a simple PrintWriter, otherwise use JDOM.

<a href="" target="_blank" rel="nofollow"></a>
I agree. Here's the link:
subject: Tackling large size XML files
It's not a secret anymore!