• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Storing XML in Filesystem or Database - Which is faster?

 
Ranch Hand
Posts: 10198
3
Mac PPC Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Guys,

I have a requirement wherein I would have to store some XML files. Now I would like to know which approach would be faster? Either storing them in the Oracle database as XML files or storing them in the File System? There would be as many as 10,000 XML files with each file half an MB of size. Let me know your suggestions?
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
My first question would be - what is the mechanism by which you locate the correct document?

If it is really simple and you are handed a file name, then the file system wins.

My next question would be - how much does this collection change over time?

Bill

 
Joe San
Ranch Hand
Posts: 10198
3
Mac PPC Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes, I have the file name and the URL stored in the database and the URL is the actual URL that points to the file which is located in the File System. The change could be much dynamic and it happens frequently. Each one of those XML files would be edited, deleted and may be new version created and so on. There would be about 5000 such folders and the mit is each folder should be capable of holding 10000 such XML files.
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Seems to me that going through the database to get the actual XML document is bound to be slower than using the provided URL directly. Furthermore it will tie up a database connection for a potentially long time.

What does Oracle recommend?

Bill
 
Ranch Hand
Posts: 862
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I am not sure which approach is faster, however either way I would try to hide which approach you used as an implementation detail. This will allow you to change approaches or maybe even come up with a better approach in the future.
 
Joe San
Ranch Hand
Posts: 10198
3
Mac PPC Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

steve souza wrote:I am not sure which approach is faster, however either way I would try to hide which approach you used as an implementation detail. This will allow you to change approaches or maybe even come up with a better approach in the future.



Fine. But I'm looking for some ideas on how the performance would be with respect to the two approaches I have mentioned here.
 
author & internet detective
Posts: 41860
908
Eclipse IDE VI Editor Java
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jothi Shankar Kumar wrote:

steve souza wrote:I am not sure which approach is faster, however either way I would try to hide which approach you used as an implementation detail. This will allow you to change approaches or maybe even come up with a better approach in the future.



Fine. But I'm looking for some ideas on how the performance would be with respect to the two approaches I have mentioned here.


William already provided this. The file system is likely to be faster since there are less steps to get it and you don't have the overhead of getting the data from the database.
 
Joe San
Ranch Hand
Posts: 10198
3
Mac PPC Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
So, here we are talking about the Java File IO system to be faster than the Java JDBC Connectivity to read and write XML data files. Aren't we?
 
Joe San
Ranch Hand
Posts: 10198
3
Mac PPC Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
In essence you guys say that Java File IO to be faster than Java JDBC?
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
YES because there is nothing magic about JDBC - the database would still have to open the file and stream the contents. Furthermore it would be tying up a database connection. By using the file system you can take advantage of any caching and buffering the operating system may be capable of.

Bill
 
Greenhorn
Posts: 9
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you are talking about the perforance of XML query (xpath etc), then file system and database are no the important factor.. the most important factor is the efficiency of XML parser xpath and indexing engine... I recommend http://vtd-xml.sf.netfor this...
if you are just talking about performance of saving xml to database or file system... there should be little difference at teh most basic level..
 
Joe San
Ranch Hand
Posts: 10198
3
Mac PPC Eclipse IDE Ubuntu
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Jamie Zhang wrote:If you are talking about the perforance of XML query (xpath etc), then file system and database are no the important factor.. the most important factor is the efficiency of XML parser xpath and indexing engine... I recommend http://vtd-xml.sf.netfor this...
if you are just talking about performance of saving xml to database or file system... there should be little difference at teh most basic level..



Exactly I'm talking about the performace of saving and retrieving XML files from the File System or the Database. Oracle supports storing XML data type directly into the database. So does this still mean that when I want to retrieve this XML datatype from the database, it opens it as a File, streams it and then puts it in the ResultSet?
 
Don't get me started about those stupid light bulbs.
reply
    Bookmark Topic Watch Topic
  • New Topic