File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes XML and Related Technologies and the fly likes XMl reading and taking huge time Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » XML and Related Technologies
Bookmark "XMl reading and taking huge time " Watch "XMl reading and taking huge time " New topic
Author

XMl reading and taking huge time

Abhi Venu
Ranch Hand

Joined: Jul 09, 2009
Posts: 73
Hai
I have written an XML reading program .It reads file and perform DB operations.I am using a SAX Parser. BUT when i run bigger files ie a 28 MB file it consumes huge amount of time.Do any have an idea what could be going wrong. What is the average time expected for reading bigger files using SAX parser?


A table, a chair, a bowl of fruit and a violin; what else does a man need to be happy?:Einstein
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12825
    
    5
I would bet real money that the main time consumer has to do with DB operations AND/OR with creation of unneeded objects rather than the parser. However, there is not need to speculate, get some monitoring software and measure.

JAMon is a likely toolkit for the purpose.

Bill
Prabhakar Reddy Bokka
Ranch Hand

Joined: Jul 26, 2005
Posts: 193

28 MB is a huge file. It takes time to read the file itself into the memory. Then your parser will start parse xml file. Thats where its consuming time.

Try to use file streams to read the file. Hope that should help.


SCJP 5, SCWCD 5
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12825
    
    5
It takes time to read the file itself into the memory. Then your parser will start parse xml file.


Alas, you are incorrect. The OP specifies the SAX approach which reads a stream and simply buffers in chunks of text. Note that this buffering is the reason the characters() method can not be relied on to get all of the characters in a TEXT node in a single call.

Bill
Abhi Venu
Ranch Hand

Joined: Jul 09, 2009
Posts: 73
I Have a 27 MB file .It takes about 3 hours to complete the operations.It actually performs some DB operations insert /Update operations in between i.e after reading certain tags.Am i doing any thing wrong or is the natural processing time required for such bigger files any way can i speed up it.
I am using SAX parser.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12825
    
    5
And have you followed my original advice and gotten some sort of monitoring tool to find out where the program is spending time?


No? Then why do you expect a helpful answer?


Lester Burnham
Rancher

Joined: Oct 14, 2008
Posts: 1337
I'd start by finding out whether it's the DB operations or the local operations that take up most of the time; JAMon can help with that.
Abhi Venu
Ranch Hand

Joined: Jul 09, 2009
Posts: 73
Hi ,

How can i learn usage of JAMon. I wrote a small prog and tested it with Monitor after adding necessary JARS i have deployed the war in TOMCat also. can i see th report for the exceution of this through admin page .
.

Thanks
Abhi
Jimmy Clark
Ranch Hand

Joined: Apr 16, 2008
Posts: 2187
28 MB XML file is actually very small. An efficiently written SAX-based application should be able to process a small file like this one in less than sixty seconds...
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: XMl reading and taking huge time