File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes XML and Related Technologies and the fly likes Parsing XML file in Chunks. Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Java 8 in Action this week in the Java 8 forum!
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "Parsing XML file in Chunks." Watch "Parsing XML file in Chunks." New topic
Author

Parsing XML file in Chunks.

Eshwar Prasad
Ranch Hand

Joined: Mar 21, 2008
Posts: 202
I have parsed the huge XML file using STAX and loaded the information in my POJO's. I used these POJO and committed data to database.

Now, I have a requirement to parse the huge XML file in Chunks. Meaning, parse partially to a threshold limit and commit to database and then again parse the left out. Can we do this using STAX API.

For example, I have 50K entries and want to set threshold limit for parse for 500. In this case, can i parse 500 entries and store in database and then continue parsing remaining.

Please advise.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12683
    
    5
SAX or Stax is exactly what you want then. Since the parse only deals with one Node at a time, it takes very little memory.

The down side will be somewhat more complex programming for you.

Beware the trap of thinking that the characters() method grabs a complete text node.

Bill

Java Resources at www.wbrogden.com
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Parsing XML file in Chunks.
 
Similar Threads
UTF8 Encoding While Writing in File - Out Of MemoryError
breaking a huge xml file into multiple small xml files
Parse XML using StaX insert into Hsql DB use maven built
How to output to an xml file after parsing
parsing huge xml files