aspose file tools*
The moose likes JDBC and the fly likes File instead of Database table Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Databases » JDBC
Bookmark "File instead of Database table" Watch "File instead of Database table" New topic
Author

File instead of Database table

Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
Just wondering whether it is possible to load the table(from the DB) contents into a file once and use the data for later use . That is extracting the data from the file instead of Extracting it from the database everytime.


Raghav.
jason adam
Chicken Farmer ()
Ranch Hand

Joined: May 08, 2001
Posts: 1932
Sure it's possible, you just have to write the file out in a way that the application knows how to pull from. Not sure how efficient that is, but it's definitely doable.
Maybe instead you could create a holder object(s) that keeps the data from the database and use that for access and manipulation, and when you're done write it back to the database. It's a lot quicker to access things in memory than in the file system.
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
I,am using object arrays to store the items fethched from the database.I am working on MVC architechture. I don't what will be the implication when the amount of data increases . Will my Array objects be able to hold such and amount of data( say 10000 rows) ??
That is th reason i was thinking of an alternative solution of storing data in a file and access it .
What do you say ?
[ March 23, 2004: Message edited by: raghav mathur ]
jason adam
Chicken Farmer ()
Ranch Hand

Joined: May 08, 2001
Posts: 1932
I wouldn't use an array, I would use some sort of Collection like an ArrayList instead. Personal preference, and keeps you from being confined to a set size.
Avoid writing to the file, especially with that amount of data. You're I/O would start to get very costly.
Depending on your database implementation, it may already be keeping things in memory so pulling from the database on a semi-regular basis might be the best solution. Holding that many objects in memory with a collection might be duplicating the effort.
Perhaps giving a flow of the application would be helpful, as well as how this application is being implemented (what type of application server, etc.) Might look into EJB's if it is applicable.
But bottom line, I wouldn't write that data to a file, the I/O stuff can be the least resource friendly way to do things.
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
The Application server is Jboss Tomact . The concept is an EJB concept (Object Arrays).The Application server is Jboss Tomact . The concept is an EJB concept (Object Arrays). Since it is a web application , Connecting it to the database often reduces its speed , especially when it comes to MIS reports.
that was the reason i thought of object arrays . Does it have a limitation?
Stefan Wagner
Ranch Hand

Joined: Jun 02, 2003
Posts: 1923

The array-index is limited to MAX_INTEGER.
And the physical memory is of course a limitation.
I wouldn't use a file, if the data changes frequently.
If you have 10,000 rows, and delete the first, you have to write 9,999 files.
Is your application the only source of changes?
Else: How do you handle changes from another source?
Why is the database-connection slow?
Do you always create new connections/ statements?
This is often the cause of slow db-performance. (Perhaps better forums: jdbc, performance).
If your handling of the jdbc-connection is 'best practice', perhaps the database-settup is semi-optimal? Are there indexes on the critical columns?
Perhaps you can talk to the database-administrator.
There are a lot of problems, database are invented for, to solve them.
Handling datapersistence, security, access, concurrency, searches, inserts, deletes, ... on your own might lead to reinvent databases.


http://home.arcor.de/hirnstrom/bewerbung
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641

I wouldn't use a file, if the data changes frequently.
If you have 10,000 rows, and delete the first, you have to write 9,999 files.

If storing each row in a file is the only solution then it is definately not a good idea .. i would agree on that.


Why is the database-connection slow?
Do you always create new connections/ statements?

The database connection is not slow i would say but if i encounter a situation where i need to make chlid-connections while looping through the parent connections then it definately reduces the load time of my JSp page.The net effect is that i am making a coneection for each row as i print a certain value.Yes i can use joins but to a certain limit.
jason adam
Chicken Farmer ()
Ranch Hand

Joined: May 08, 2001
Posts: 1932
And since we've moved into efficient use of JDBC connections, I think I will send this to that forum. I think we all pretty much agree writing to a file is a bad thing to do
Stefan Wagner
Ranch Hand

Joined: Jun 02, 2003
Posts: 1923

And please excuse my attention-error; of course you don't have to write 9.999 files but rows.
How many connections to you have at once?
I've heard of people making one connection per statement, but one connection per row?
Raghav Mathur
Ranch Hand

Joined: Jan 12, 2001
Posts: 641
Originally posted by Stefan Wagner:

How many connections to you have at once?
I've heard of people making one connection per statement, but one connection per row?

Yes. There comes a situation when i am left with no choice but to use another connection to retrieve a piece of data . That means when i am looping through the recorset and retireving each row , i am also making a connection to retrieve some information related to the information i am retrieving through the recordset.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: File instead of Database table
 
Similar Threads
Is Deployment Descriptor always required?
How do I create an Object Pool?
File instead of Database table
back up database
Export data to Excel in Websphere Portals