aspose file tools*
The moose likes Performance and the fly likes Storing XML in Filesystem or Database - Which is faster? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Java » Performance
Bookmark "Storing XML in Filesystem or Database - Which is faster?" Watch "Storing XML in Filesystem or Database - Which is faster?" New topic
Author

Storing XML in Filesystem or Database - Which is faster?

Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9427
    
    2

Guys,

I have a requirement wherein I would have to store some XML files. Now I would like to know which approach would be faster? Either storing them in the Oracle database as XML files or storing them in the File System? There would be as many as 10,000 XML files with each file half an MB of size. Let me know your suggestions?


SCJP 1.4, SCWCD 1.4 - Hints for you, Certified Scrum Master
Did a rm -R / to find out that I lost my entire Linux installation!
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12806
    
    5
My first question would be - what is the mechanism by which you locate the correct document?

If it is really simple and you are handed a file name, then the file system wins.

My next question would be - how much does this collection change over time?

Bill

Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9427
    
    2

Yes, I have the file name and the URL stored in the database and the URL is the actual URL that points to the file which is located in the File System. The change could be much dynamic and it happens frequently. Each one of those XML files would be edited, deleted and may be new version created and so on. There would be about 5000 such folders and the mit is each folder should be capable of holding 10000 such XML files.
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12806
    
    5
Seems to me that going through the database to get the actual XML document is bound to be slower than using the provided URL directly. Furthermore it will tie up a database connection for a potentially long time.

What does Oracle recommend?

Bill
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 861
I am not sure which approach is faster, however either way I would try to hide which approach you used as an implementation detail. This will allow you to change approaches or maybe even come up with a better approach in the future.


http://www.jamonapi.com/ - a fast, free open source performance tuning api.
JavaRanch Performance FAQ
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9427
    
    2

steve souza wrote:I am not sure which approach is faster, however either way I would try to hide which approach you used as an implementation detail. This will allow you to change approaches or maybe even come up with a better approach in the future.


Fine. But I'm looking for some ideas on how the performance would be with respect to the two approaches I have mentioned here.
Jeanne Boyarsky
author & internet detective
Marshal

Joined: May 26, 2003
Posts: 30764
    
156

Jothi Shankar Kumar wrote:
steve souza wrote:I am not sure which approach is faster, however either way I would try to hide which approach you used as an implementation detail. This will allow you to change approaches or maybe even come up with a better approach in the future.


Fine. But I'm looking for some ideas on how the performance would be with respect to the two approaches I have mentioned here.

William already provided this. The file system is likely to be faster since there are less steps to get it and you don't have the overhead of getting the data from the database.


[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9427
    
    2

So, here we are talking about the Java File IO system to be faster than the Java JDBC Connectivity to read and write XML data files. Aren't we?
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9427
    
    2

In essence you guys say that Java File IO to be faster than Java JDBC?
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12806
    
    5
YES because there is nothing magic about JDBC - the database would still have to open the file and stream the contents. Furthermore it would be tying up a database connection. By using the file system you can take advantage of any caching and buffering the operating system may be capable of.

Bill
Jamie Zhang
Greenhorn

Joined: Jul 31, 2009
Posts: 9
If you are talking about the perforance of XML query (xpath etc), then file system and database are no the important factor.. the most important factor is the efficiency of XML parser xpath and indexing engine... I recommend http://vtd-xml.sf.netfor this...
if you are just talking about performance of saving xml to database or file system... there should be little difference at teh most basic level..
Joe Harry
Ranch Hand

Joined: Sep 26, 2006
Posts: 9427
    
    2

Jamie Zhang wrote:If you are talking about the perforance of XML query (xpath etc), then file system and database are no the important factor.. the most important factor is the efficiency of XML parser xpath and indexing engine... I recommend http://vtd-xml.sf.netfor this...
if you are just talking about performance of saving xml to database or file system... there should be little difference at teh most basic level..


Exactly I'm talking about the performace of saving and retrieving XML files from the File System or the Database. Oracle supports storing XML data type directly into the database. So does this still mean that when I want to retrieve this XML datatype from the database, it opens it as a File, streams it and then puts it in the ResultSet?
 
 
subject: Storing XML in Filesystem or Database - Which is faster?