• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

XML Conversion of millions of records

 
Renjith Panikar
Greenhorn
Posts: 28
Java Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi All,

I am working on an application in which i have to convert the contents of a text file to xml format.

Text file contains millions of records. Format is given below.

Header1
Data of Header1 (1)
.
.
Data of Header1(n)
Header2
Data of Header2 (1)
.
.
Data of Header2(n)
etc...

This has to be converted to XML of format

<ItemInfo>
<Item>
<Header>
<Header1>
</Header>
<Data>
<Data(1)>........<Data(n)>
</Data>
<Item>
<Item>
<Header>
<Header2>
</Header>
<Data>
<Data(1)>........<Data(n)>
</Data>
<Item>
<ItemInfo>

Each header and it data from 1-n create a single record.
Since there can be millions of records, we cannot take entire records and convert it as XML.
Currently it fetches first 100 records , convert it into XML then fetch next 100 and so on.
XML decleration and <ItemInfo> tags are generated using stax parser. records are converted
tom<Item> using jaxb. But it is taking long time to execute. eg. 4 minutes to convert 1000 records in windows machine
with 4 gb memory.

The reason why i used stax is that, i donno how to insert <Item> inside <ItemInfo> tags after creating <Iteminfo > </ItemInfo>
tags using Jaxb.

Can you suggest me a better solution for this issue.

Is it possible to use jaxb alone to do this?

thanks in advance.

renjith




 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic