File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes XML and Related Technologies and the fly likes XML Conversion of millions of records Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "XML Conversion of millions of records" Watch "XML Conversion of millions of records" New topic
Author

XML Conversion of millions of records

Renjith Panikar
Greenhorn

Joined: Nov 06, 2012
Posts: 11
Hi All,

I am working on an application in which i have to convert the contents of a text file to xml format.

Text file contains millions of records. Format is given below.

Header1
Data of Header1 (1)
.
.
Data of Header1(n)
Header2
Data of Header2 (1)
.
.
Data of Header2(n)
etc...

This has to be converted to XML of format

<ItemInfo>
<Item>
<Header>
<Header1>
</Header>
<Data>
<Data(1)>........<Data(n)>
</Data>
<Item>
<Item>
<Header>
<Header2>
</Header>
<Data>
<Data(1)>........<Data(n)>
</Data>
<Item>
<ItemInfo>

Each header and it data from 1-n create a single record.
Since there can be millions of records, we cannot take entire records and convert it as XML.
Currently it fetches first 100 records , convert it into XML then fetch next 100 and so on.
XML decleration and <ItemInfo> tags are generated using stax parser. records are converted
tom<Item> using jaxb. But it is taking long time to execute. eg. 4 minutes to convert 1000 records in windows machine
with 4 gb memory.

The reason why i used stax is that, i donno how to insert <Item> inside <ItemInfo> tags after creating <Iteminfo > </ItemInfo>
tags using Jaxb.

Can you suggest me a better solution for this issue.

Is it possible to use jaxb alone to do this?

thanks in advance.

renjith




 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: XML Conversion of millions of records
 
Similar Threads
convert Excel to XML format
Regex Servlet issue
Double line column header problem in SWT
column width in rich:datatable
Header, Footer with dynamic data