• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

XMl reading and taking huge time

 
Abhi Venu
Ranch Hand
Posts: 73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hai
I have written an XML reading program .It reads file and perform DB operations.I am using a SAX Parser. BUT when i run bigger files ie a 28 MB file it consumes huge amount of time.Do any have an idea what could be going wrong. What is the average time expected for reading bigger files using SAX parser?
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13058
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I would bet real money that the main time consumer has to do with DB operations AND/OR with creation of unneeded objects rather than the parser. However, there is not need to speculate, get some monitoring software and measure.

JAMon is a likely toolkit for the purpose.

Bill
 
Prabhakar Reddy Bokka
Ranch Hand
Posts: 197
Java Oracle Spring
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
28 MB is a huge file. It takes time to read the file itself into the memory. Then your parser will start parse xml file. Thats where its consuming time.

Try to use file streams to read the file. Hope that should help.
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13058
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It takes time to read the file itself into the memory. Then your parser will start parse xml file.


Alas, you are incorrect. The OP specifies the SAX approach which reads a stream and simply buffers in chunks of text. Note that this buffering is the reason the characters() method can not be relied on to get all of the characters in a TEXT node in a single call.

Bill
 
Abhi Venu
Ranch Hand
Posts: 73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I Have a 27 MB file .It takes about 3 hours to complete the operations.It actually performs some DB operations insert /Update operations in between i.e after reading certain tags.Am i doing any thing wrong or is the natural processing time required for such bigger files any way can i speed up it.
I am using SAX parser.
 
William Brogden
Author and all-around good cowpoke
Rancher
Posts: 13058
6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
And have you followed my original advice and gotten some sort of monitoring tool to find out where the program is spending time?


No? Then why do you expect a helpful answer?


 
Lester Burnham
Rancher
Posts: 1337
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'd start by finding out whether it's the DB operations or the local operations that take up most of the time; JAMon can help with that.
 
Abhi Venu
Ranch Hand
Posts: 73
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi ,

How can i learn usage of JAMon. I wrote a small prog and tested it with Monitor after adding necessary JARS i have deployed the war in TOMCat also. can i see th report for the exceution of this through admin page .
.

Thanks
Abhi
 
Jimmy Clark
Ranch Hand
Posts: 2187
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
28 MB XML file is actually very small. An efficiently written SAX-based application should be able to process a small file like this one in less than sixty seconds...
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic