aspose file tools*
The moose likes Threads and Synchronization and the fly likes FixedThreadPool issue: more threads then expected seems to be working Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "FixedThreadPool issue: more threads then expected seems to be working" Watch "FixedThreadPool issue: more threads then expected seems to be working" New topic
Author

FixedThreadPool issue: more threads then expected seems to be working

hildich kilal
Ranch Hand

Joined: Aug 25, 2008
Posts: 44
Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich
Henry Wong
author
Sheriff

Joined: Sep 28, 2004
Posts: 18896
    
  40

hildich kilal wrote:Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich



The fixed thread pool is exactly that -- it doesn't create new thread pool threads. So, I would recommend looking at your callable again.


However, if you don't believe that this is so (ie. you disagree), then do a thread dump to check how many threads are in the thread pool.

Henry


Books: Java Threads, 3rd Edition, Jini in a Nutshell, and Java Gems (contributor)
Ranganathan Kaliyur Mannar
Bartender

Joined: Oct 16, 2003
Posts: 1085
    
  10

A FixedThreadPool will create a max of n threads at a given point. But, if you submit more tasks, once earlier threads are done and the current number of threads become less than n, then it will create further threads for those tasks.

This is what is said in the API:
"At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available."

So, if you haven't closed the connection properly, then there is a possibility many connections being open...


Ranga.
SCJP 1.4, OCMJEA/SCEA 5.0.
Martin Vajsar
Sheriff

Joined: Aug 22, 2010
Posts: 3610
    
  60

I'd just mention that to effectively process 20 requests in your database concurrently, the database must be sized up to the task. If it isn't, the processing might actually take longer and squeeze out other users from the database.

That said, moving the processing to the database might be much, much more effective to shorten the processing time.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: FixedThreadPool issue: more threads then expected seems to be working