This week's book giveaway is in the Java 8 forum.
We're giving away four copies of Java 8 in Action and have Raoul-Gabriel Urma, Mario Fusco, and Alan Mycroft on-line!
See this thread for details.
The moose likes JDBC and the fly likes Handling large amount of data in tomcat memory Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Java 8 in Action this week in the Java 8 forum!
JavaRanch » Java Forums » Databases » JDBC
Bookmark "Handling large amount of data in tomcat memory" Watch "Handling large amount of data in tomcat memory" New topic
Author

Handling large amount of data in tomcat memory

Manish Chhetri
Greenhorn

Joined: Oct 31, 2003
Posts: 11
Hi

I have a really unique problem with this one..I have a database having 19 tables each of which contains more than 50,000 records and data keeps flowing to these tables (their size keeps increasing). I have joins on these tables. Now i have a web interface through which these data can be viewed. For each user logged in when i get records, the tomcat memory is used..to keep this data. After running this application for 1 day, tomcat gives an out of memory exception. How do i handle this memory problem.

PS: the users are not allowed to view all records. Different users have different rights.


The One..
satish sathineni
Ranch Hand

Joined: May 03, 2004
Posts: 46
Hi,

Manish

I also faced the same solution for longtime i was able to solve it..

first i would like to know the OS ur running the tomcat and version of tomcat webserver....

regards
satish kumar
Julian Kennedy
Ranch Hand

Joined: Aug 02, 2004
Posts: 823
Hi Manish,

That problem doesn't sound so unique to me; it's an old, old problem. Firstly, are you sure you don't have a memory leak? If you're confident that you don't then (and perhaps anyway) the solution is to implement some sort of caching such that there is a limit on the amount of memory the system will use (that you set). If that memory limit is reached you need some policy for clearing out space (e.g. least recently used data).

Hope that helps.

Jules
Manish Chhetri
Greenhorn

Joined: Oct 31, 2003
Posts: 11
Hi Guys...

I am using windows 2000 OS. How will it matter.
Julian Kennedy
Ranch Hand

Joined: Aug 02, 2004
Posts: 823
AFAIK, it wouldn't.
Vikalp Singh
Ranch Hand

Joined: Dec 29, 2002
Posts: 50
Even we are faced same problem in our application.
First of all check for the memory leak, and try to reduce the data loading on the basis of some paging rule.
And if you are using Rowset for data load, then try to use CachedrowSet.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Handling large amount of data in tomcat memory
 
Similar Threads
automatic dirty checking
doing a dynamic thing in xslt
Is this a good idea
Problems changing foreign key references in Grails?
Performance issue - selecting data from a large table