aspose file tools*
The moose likes XML and Related Technologies and the fly likes What to use when Applying XSL to 150Mb XML file Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "What to use when Applying XSL to 150Mb XML file" Watch "What to use when Applying XSL to 150Mb XML file" New topic
Author

What to use when Applying XSL to 150Mb XML file

raje malhotras
Greenhorn

Joined: Oct 31, 2007
Posts: 3
Hi
I have a serious problem
I am converting very large XML(greater than 150 Mb) to another XML by using XSLT transformation in Java 5.
But in this process jvm uses a very large amount of memory(heap of jvm).

I have use XALAN and java inbuilt parsers for this transformation but still memory usage is very high ....

So i am searching for some parsers or something like that..... by which when i do this transformation my memory usage will be as low as posssible.........

so is there any way to do my memory usage minimum.....

Thanks for any help
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12806
    
    5
If there is any way you can pre-process the monster XML file to remove unused data or re-arrange into a more logical order it will reduce the XSLT memory requirement.

How complex is the XSLT logic?

Bill
raje malhotras
Greenhorn

Joined: Oct 31, 2007
Posts: 3
XSLT logic is not complex .
It just repaces tag names and something like that.

The main problem is size of XML Because of which memory requirement is too high.......

Any suggestions are also invited.......
raje malhotras
Greenhorn

Joined: Oct 31, 2007
Posts: 3
There is no scope of pre processing the XML file .If i do so ,in that case the size of XML is still greater than 150Mb.

XSLT logic is not complex .
It just repaces tag names and something like that.

The main problem is size of XML Because of which memory requirement is too high.......

But i wanted to reduce memory requirement..............
Any suggestions are also invited.......
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18662
    
    8

It doesn't matter how complex or simple your XSLT logic is, an XSLT processor will almost always load the entire XML document into memory before applying that logic. So your best bet is really to not get involved in designs that include giant XML documents. Not if you have to apply XSLT to them, that is.

However I believe that the SAXON transformer does apply some optimizations for XPath expressions that can be proven to be (I forget the exact term) serial. This means that you can apply the expression to a document while reading the document from beginning to end without having to back up, and it means that the transformer doesn't have to build a document tree in memory.

You might look into this, although I predict that as soon as you decide to use it, somebody will right away change the XSLT to something else simple where this optimization can't be applied, and you are right back to where you started.

Good luck.
Amit Arora
Ranch Hand

Joined: Jun 14, 2007
Posts: 30
Hi

You could try for STX to be used for XML transformation. It does not load the document in the memory. The heap size would be less. But it is a different language used for transformation.


Amit Arora<br />SCJP 1.4, SCWCD
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: What to use when Applying XSL to 150Mb XML file