• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Liutauras Vilda
  • Paul Clapham
  • Bear Bibeault
  • Jeanne Boyarsky
Sheriffs:
  • Ron McLeod
  • Tim Cooke
  • Devaka Cooray
Saloon Keepers:
  • Tim Moores
  • Tim Holloway
  • Jj Roberts
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • salvin francis
  • Scott Selikoff
  • fred rosenberger

Process Huge XML documents using VTD XML parser

 
Ranch Hand
Posts: 70
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

we came across the VTD XML Parser which seems to be very cool to parse XML files and we decided to use the VTD-XML Huge API to parse an xml file which is 3GB in size. we followed the documentation given in the below link

http://vtd-xml.sourceforge.net/codeSample/cs12.html

For some reason the VTD Huge API seems to fail while parsing and doesnt seem to be able to complete parsing the file, we were told that there could be some problem with the API as it was still in beta version. If there is anyone who was in fact able to use the API for parsing huge xml files, could you please let me know in case of any known tweaks or issues.
 
Marshal
Posts: 25949
69
Eclipse IDE Firefox Browser MySQL Database
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Seems to me your first step should be to find out if you actually have a problem. You said

For some reason the VTD Huge API seems to fail while parsing and doesnt seem to be able to complete parsing the file...


which suggests that you don't quite know whether the code worked or not. So first find out whether it's working or not. Does it produce the output you expected? Or does it throw an exception? If there's a problem, it may be as simple as not setting your classpath correctly, so asking people to provide solutions for problems which may not be your problem is really a waste of time for all concerned.
 
Rajesh MadhanGopal
Ranch Hand
Posts: 70
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hi,

May be i wasnt clear enough...we tried the normal VTD API for a file which was 600MB size and it seemed to work fine, since the VTD HUGE parser was meant for files bigger than 2 GB - we tried a file which was 3GB and it did not work fine. We even tried a smaller file (500 MB) with VTD Huge and it did not seem to work fine. The code was updated correctly for the VTD HUGE parser and also the Jars required for VTD HUGE were included in the classpath, may be i am missing something which iam unable to figure out as yet..

But the question was : If there was a known issue with VTD Huge parser atall as some forums hinted the same....
 
Bartender
Posts: 1210
25
Android Python PHP C++ Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Rajesh MadhanGopal wrote:...600MB size and it seemed to work fine....3GB and it did not work fine



Hi,

The post is rather vague. Are you getting an exception? Please TellTheDetails because ItDoesntWorkIsUseless.
 
When people don’t understand what you are doing they call you crazy. But this tiny ad just doesn't care:
the value of filler advertising in 2020
https://coderanch.com/t/730886/filler-advertising
reply
    Bookmark Topic Watch Topic
  • New Topic