• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

XMl reading and taking huge time

 
Ranch Hand
Posts: 73
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hai
I have written an XML reading program .It reads file and perform DB operations.I am using a SAX Parser. BUT when i run bigger files ie a 28 MB file it consumes huge amount of time.Do any have an idea what could be going wrong. What is the average time expected for reading bigger files using SAX parser?
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I would bet real money that the main time consumer has to do with DB operations AND/OR with creation of unneeded objects rather than the parser. However, there is not need to speculate, get some monitoring software and measure.

JAMon is a likely toolkit for the purpose.

Bill
 
Ranch Hand
Posts: 198
Oracle Spring Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
28 MB is a huge file. It takes time to read the file itself into the memory. Then your parser will start parse xml file. Thats where its consuming time.

Try to use file streams to read the file. Hope that should help.
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

It takes time to read the file itself into the memory. Then your parser will start parse xml file.



Alas, you are incorrect. The OP specifies the SAX approach which reads a stream and simply buffers in chunks of text. Note that this buffering is the reason the characters() method can not be relied on to get all of the characters in a TEXT node in a single call.

Bill
 
Abhi Venu
Ranch Hand
Posts: 73
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I Have a 27 MB file .It takes about 3 hours to complete the operations.It actually performs some DB operations insert /Update operations in between i.e after reading certain tags.Am i doing any thing wrong or is the natural processing time required for such bigger files any way can i speed up it.
I am using SAX parser.
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
And have you followed my original advice and gotten some sort of monitoring tool to find out where the program is spending time?


No? Then why do you expect a helpful answer?


 
Rancher
Posts: 1337
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'd start by finding out whether it's the DB operations or the local operations that take up most of the time; JAMon can help with that.
 
Abhi Venu
Ranch Hand
Posts: 73
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi ,

How can i learn usage of JAMon. I wrote a small prog and tested it with Monitor after adding necessary JARS i have deployed the war in TOMCat also. can i see th report for the exceution of this through admin page .
.

Thanks
Abhi
 
Ranch Hand
Posts: 2187
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
28 MB XML file is actually very small. An efficiently written SAX-based application should be able to process a small file like this one in less than sixty seconds...
 
Your mother was a hamster and your father was a tiny ad:
a bit of art, as a gift, that will fit in a stocking
https://gardener-gift.com
reply
    Bookmark Topic Watch Topic
  • New Topic