• Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

FixedThreadPool issue: more threads then expected seems to be working

 
hildich kilal
Ranch Hand
Posts: 44
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich
 
Henry Wong
author
Marshal
Pie
Posts: 20828
75
C++ Chrome Eclipse IDE Firefox Browser Java jQuery Linux VI Editor Windows
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
hildich kilal wrote:Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich



The fixed thread pool is exactly that -- it doesn't create new thread pool threads. So, I would recommend looking at your callable again.


However, if you don't believe that this is so (ie. you disagree), then do a thread dump to check how many threads are in the thread pool.

Henry
 
Ranganathan Kaliyur Mannar
Bartender
Posts: 1101
10
Java Netbeans IDE Ubuntu
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
A FixedThreadPool will create a max of n threads at a given point. But, if you submit more tasks, once earlier threads are done and the current number of threads become less than n, then it will create further threads for those tasks.

This is what is said in the API:
"At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available."

So, if you haven't closed the connection properly, then there is a possibility many connections being open...
 
Martin Vajsar
Sheriff
Pie
Posts: 3747
62
Chrome Netbeans IDE Oracle
  • 0
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'd just mention that to effectively process 20 requests in your database concurrently, the database must be sized up to the task. If it isn't, the processing might actually take longer and squeeze out other users from the database.

That said, moving the processing to the database might be much, much more effective to shorten the processing time.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic