This week's giveaway is in the Android forum.
We're giving away four copies of Android Security Essentials Live Lessons and have Godfrey Nolan on-line!
See this thread for details.
The moose likes JDBC and the fly likes Memory leak with MySQL JDBC driver Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Databases » JDBC
Bookmark "Memory leak with MySQL JDBC driver" Watch "Memory leak with MySQL JDBC driver" New topic
Author

Memory leak with MySQL JDBC driver

Henrik Karlsson
Greenhorn

Joined: Oct 19, 2006
Posts: 1
I have a problem with memory usage on my application. A short introduction to my app:
The application collects data from a number of MySQL servers (we call these dbs external) and puts the data into one single database (MySQL, we call this db local). The collection should be done periodically. These two classes are of intrest:
- Fetcher. Upon creation of this class it sets up a connection to one of the external servers. This connection will be open until the application terminates. Fetcher has a method, fetch(), which executes a query and gets a ResultSet from the database.


- FetcherThread extends Thread. Dont think this will need any explanation.

Since Fetcher.fetch only is accessed by one thread (there is one FetcherThread and one Fetcher for each external database) there shouldnt be any special real time requirements here. Everything concerning the local database can be ignored at this point, it has nothing to do with my problem (same shit happens when qh.updateAlarms() is commented).

So to my problem. The code above eats memory like crazy (for one extrenal database, i.e. one Fetcher and one FetcherThread, 1mb each 10 minutes), even if I just execute the query and then ignore the ResultSet.
The result returned from the query are at max 10-15 rows.
I've tried setting everything to null in the end of Fetcher.fetch(), calling alarms.removeAllElements(), replacing Vector with LinkedList and explicitly call System.gc() but with no success.

I did some profiling (with TPTP), and it seems like the GC misses to collect about 50% of the vectors elemets (instances of Alarm). The Alarm class simply is a container, and has no references to other objects. But as I mentioned above, even with this part removed it leaks memory.

btw. the constant SMTDataCollector.ALARM_FETCH_DELAY = 3000, i.e. the thread will sleep for 3 seconds.

Any thoughts about this are very appriciated.
Tim LeMaster
Ranch Hand

Joined: Aug 31, 2006
Posts: 226
System.gc() just suggests the garbage collector run. It may or may not run. Also when it runs it doesn't have to collect all eligible objects.

At first glance I would say you aren't calling close() on your Statement or ResultSet so these will not be closed until the garbage collector collects them.

I'd call close() or your statement before exiting fetch().
George Stoianov
Ranch Hand

Joined: Jan 15, 2006
Posts: 94
Besides the good comments made about closing stuff I have a few other questions:
1. Where does con come from?
2. Do all dbs have the same structure and that is why you can just pass different connections and collect data, I assume?
3. Where is sequenceNbr defined? I know you said only one thread per fetcher but putting some printouts or logs would be interesting to see.
4. If gc is not collecting 50% of Vector elements it maybe doing so for optimization lets say it was keeping a pool and reusing values, are this strings in binary??, as you do not seem to be reassigning individual elements somewhere check your qh.updateAlarms() code.
5. If nothing works start removing/rewriting code line by line, you are probably doing that , and isolate the problem inside or out of a particular block.
6. Since you have multithreading make sure your assumptions on what is happening are actually correct.

Good luck.
George
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Memory leak with MySQL JDBC driver
 
Similar Threads
Concurrency Issue with HSQLDB
web.xml
How to create sequence for my table in Oracle
Using Vectors
LOAD DATA INFILE problem