my dog learned polymorphism
The moose likes XML and Related Technologies and the fly likes Tackling large size XML files Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login

Win a copy of OCA Java SE 8 Programmer I Study Guide this week in the OCAJP 8 forum!
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "Tackling large size XML files" Watch "Tackling large size XML files" New topic

Tackling large size XML files

Anil C

Joined: Jan 08, 2001
Posts: 16
I have a requirement to port large amount of data from one DB to another via XML. We are estimating the size of the XML file (my XML includes data from more than a score tables) in the filesystem to be somewhere near 20MB. In the process of uploading, if I have to load this XML file to memory as a DOM object, then it will take up almost 40MB. (or more ?). Is there any way to avoid loading the whole XML but extract only portions ( say, table by table...Due to some technical limitations, we found the option of having separate XML files for each table not viable.)and then extract data phase by phase ? SAX and jDOM have been suggested as possible solutions...I am new to both and as there is a time constraint, will appreciate if anyone can throw more light on the best approach possible
Manuel Palacio
Ranch Hand

Joined: Oct 16, 2000
Posts: 45
I was faced with a similar problem-- a rdf file containing thousands of objects and I solved it with SAX. Use SAX to parse the file. This way you avoid creating the whole tree in memory. And to create a large document use a simple PrintWriter, otherwise use JDOM.

<a href="" target="_blank" rel="nofollow"></a>
I agree. Here's the link:
subject: Tackling large size XML files
It's not a secret anymore!