File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes XML and Related Technologies and the fly likes XSOMParser throwing out of memory error Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "XSOMParser throwing out of memory error" Watch "XSOMParser throwing out of memory error" New topic
Author

XSOMParser throwing out of memory error

skumar kk
Greenhorn

Joined: Apr 06, 2011
Posts: 6
Hi,

I am using XSOMParser to parse large xsd files. When I am calling XSOMParser.parse(Filepath) method it is throwing out of memory error.

As I know XSOMParser internally uses SAX parser to parse document and SAX parser support event based mechanism and reads input line by line. So very less memory is required to parse large files. Correct me if wrong.

Please let me know if any one has worked on similar issue.

code :

XSOMParser parser = new XSOMParser();
parser.setAnnotationParser(new DomAnnotationParserFactory());

try {
try {
parser.parse(new File(xsdFilePath));
} catch (IOException e) {
e.printStackTrace();
}

----------
Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18570
    
    8

skumar kk wrote:As I know XSOMParser internally uses SAX parser to parse document and SAX parser support event based mechanism and reads input line by line. So very less memory is required to parse large files. Correct me if wrong.


That could be right (I'm not familiar with XSOM). Let's suppose it is. It still says nothing about whether XSOM takes the output of that SAX parser and builds some internal structure which can use up all available memory if it becomes large enough, so it isn't very helpful as an approach to your problem. Especially as there is no doubt that you are running out of memory.

But I wouldn't expect that an XML schema could be large enough to use up all available memory. Perhaps your schema is extremely large? Or perhaps there is something strange about it?
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12782
    
    5
Perhaps there is some sort of recursion going on. I always suspect recursion when small jobs throw out of memory errors.

Bill
skumar kk
Greenhorn

Joined: Apr 06, 2011
Posts: 6
William Brogden wrote:Perhaps there is some sort of recursion going on. I always suspect recursion when small jobs throw out of memory errors.


Hi me getting error on XSOMParser.parse() method where we are not using any sort of recursion. I do not have information about how parse method was internally implemented.

Stack Trace :

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at oracle.xml.parser.v2.XMLDocument.xdkIncCurrentId(XMLDocument.java:3020)
at oracle.xml.parser.v2.XMLNode.xdkInit(XMLNode.java:2758)
at oracle.xml.parser.v2.XMLNode.<init>(XMLNode.java:423)
at oracle.xml.parser.v2.XMLNSNode.<init>(XMLNSNode.java:144)
at oracle.xml.parser.v2.XMLElement.<init>(XMLElement.java:373)
at oracle.xml.parser.v2.XMLDocument.createNodeFromType(XMLDocument.java:2865)
at oracle.xml.parser.v2.XMLDocument.createElement(XMLDocument.java:1896)
at oracle.xml.parser.v2.DocumentBuilder.startElement(DocumentBuilder.java:224)
at oracle.xml.parser.v2.XMLElement.reportStartElement(XMLElement.java:3188)
at oracle.xml.parser.v2.XMLElement.reportSAXEvents(XMLElement.java:2164)
at oracle.xml.jaxp.JXTransformer.transform(JXTransformer.java:337)
at oracle.xml.jaxp.JXTransformerHandler.endDocument(JXTransformerHandler.java:141)
at com.sun.xml.xsom.impl.parser.state.NGCCRuntime.endElement(NGCCRuntime.java:267)
at org.xml.sax.helpers.XMLFilterImpl.endElement(Unknown Source)
at oracle.xml.parser.v2.NonValidatingParser.parseElement(NonValidatingParser.java:1257)
at oracle.xml.parser.v2.NonValidatingParser.parseRootElement(NonValidatingParser.java:314)
at oracle.xml.parser.v2.NonValidatingParser.parseDocument(NonValidatingParser.java:281)
at oracle.xml.parser.v2.XMLParser.parse(XMLParser.java:196)
at org.xml.sax.helpers.XMLFilterImpl.parse(Unknown Source)
at com.sun.xml.xsom.parser.JAXPParser.parse(JAXPParser.java:79)
at com.sun.xml.xsom.impl.parser.NGCCRuntimeEx.parseEntity(NGCCRuntimeEx.java:298)
at com.sun.xml.xsom.impl.parser.ParserContext.parse(ParserContext.java:87)
at com.sun.xml.xsom.parser.XSOMParser.parse(XSOMParser.java:147)
at com.sun.xml.xsom.parser.XSOMParser.parse(XSOMParser.java:136)
at com.sun.xml.xsom.parser.XSOMParser.parse(XSOMParser.java:129)
at com.sun.xml.xsom.parser.XSOMParser.parse(XSOMParser.java:122)
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12782
    
    5
I could not find any JavaDocs for this class and method:

at oracle.xml.parser.v2.XMLDocument.xdkIncCurrentId(XMLDocument.java:3020)

However, it sounds like a call to include another document - could this lead to a loop which just keeps including till it runs out of memory?

Bill

g tsuji
Ranch Hand

Joined: Jan 18, 2011
Posts: 512
    
    3
The xsom parser will certainly build its own object models in-memory (om of xsom stands for object model, does it not?). It is certainly not a sax parser. However, the AnnotationParser implements and use sax ContentHandlerm though. But what it, the AnnotationParser, builds and then returns through getResult() method can be a huge string or even use the resultant string to further parse it into dom tree (it gets that much of freedom there). Both stories can contribute to the stress on the memory usage.

I would suggest you first leave out the setAnnotationParser() part to see if it works fine in getting XSSchemaSet etc... and decide further action from there. (What is the sum of sizes of the xsd's on disk?) If that part loads well, then you've to look into your AnnotationParser implementation see what it actually does, if you really want to parse annotations as well. Since the implementation of the AnnotationParser is yours, I would not anticipate any difficulty in knowing what you want actually out of it and how big it would be of the object return through the getResult().
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: XSOMParser throwing out of memory error