aspose file tools*
The moose likes XML and Related Technologies and the fly likes Searching for a value in a large XML file Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "Searching for a value in a large XML file" Watch "Searching for a value in a large XML file" New topic
Author

Searching for a value in a large XML file

ratna sargurp
Ranch Hand

Joined: Jan 12, 2010
Posts: 32
Hello

I have a large XML file (1GB) that I have to parse through and search for content in a particular element. Is SAX better or JDOM a better approach to this problem?

Any suggestions are appreciated.

Thanks
Jelle Klap
Bartender

Joined: Mar 10, 2008
Posts: 1666
    
    7

JDOM is a framework for XML processing that supports SAX, so you can't compare the two. You'll definitely want to use either a SAX, or StAX approach to processing the file, instead of DOM (I think you confused this with JDOM).


Build a man a fire, and he'll be warm for a day. Set a man on fire, and he'll be warm for the rest of his life.
Campbell Ritchie
Sheriff

Joined: Oct 13, 2005
Posts: 36501
    
  16
Too difficult a question for “beginning”. Moving thread.
ratna sargurp
Ranch Hand

Joined: Jan 12, 2010
Posts: 32
Thank you. I started using SAX parser and was able to parse the data except now I have hit a road block. The input XML file cannot be modified at all. I can only read it. This input file has entity referenced like &rdquo that the parser does not like. The entities are not declared within the XML file. They reside in a separate file.

In order to resolve this issue, I wrote a separate entity resolver by overriding resolveEntity. Please see my code below:



And in my SAX parser code I call this resolver:

xmlReader.setEntityResolver(new CustomEntityResolver());

But this code is not working.

Please note that I have separate public ID strings for each of the entities and have only shown one here as an example.

Any tips or suggestions are appreciated.



Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18129
    
    8

If the parser is balking at ” because it isn't properly declared by a DTD in the document, then I don't see what the point is of the string "ISO 8879-1986//ENTITIES Alternative Greek Symbols//EN" in that code.

Or is that entity reference supposed to be resolved by a schema which the parser doesn't know about?

Or is the problem that there is a DTD but the parser can't access it? That would be a problem solved by your EntityResolver, so perhaps the problem isn't that.

Anyway it would help if you told us what the actual problem was, rather than trying to get us to help with a solution which might not be the solution to the actual problem.
ratna sargurp
Ranch Hand

Joined: Jan 12, 2010
Posts: 32
Thanks Paul. My issue is to be able to somehow ignore those non-xml entities while parsing the data. The two problems are that I cannot modify the XML file, it has no DTD declaration and the DTD itself resides outside the file.

That was the reason I started writing the Entity Resolver.

What is your suggestion to this issue?

Paul Clapham
Bartender

Joined: Oct 14, 2005
Posts: 18129
    
    8

Well, plainly if your XML uses those entity references but doesn't contain a DTD which declares them, then it's not a well-formed XML document. And parsers won't parse it.

Do you have any other processes which parse this document? If so, maybe you can find out how they persuade the parser to refer to the DTD.

Or perhaps you could get the people who produced that document to modify their processes so that they produce well-formed XML.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Searching for a value in a large XML file
 
Similar Threads
how to process large XML file(bigger than 1GB) in Java?
Read large xml file for searching
.jsp file to .pdf file
Recommendation on Best XML API to create XML 1.0 files
how about process large XML file(bigger than 1GB) in Java?