FixedThreadPool issue: more threads then expected seems to be working

[hildich kilal]

Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich

[Henry Wong]

hildich kilal wrote:Hello,
I am facing an issue with FixedThreadPoolExecutor.

So scenario is that I need to load some data in cache from database. For this I run JDBC queries in database.
Now I need to run those queries in parallel . To get all data quickly.

I start an "newFixedThreadPool" executorService and there is one "Callable" class , say 'X' whose "call" method makes JDBC connection and executes select
queries.
I run a loop of around 5000 and create a new 'X' and submit it to executorService.
Now what i find is that even though I set pool size to some thing say 20 , I can see more than 100 -200 connections made in DB!

I have checked that I am closing connection properly . Also no other process is making any connection.

Is there a way this executorService can do that? The connection is made in "call" and closed there only.

Please suggest

Thanks
Hildich

The fixed thread pool is exactly that -- it doesn't create new thread pool threads. So, I would recommend looking at your callable again.


However, if you don't believe that this is so (ie. you disagree), then do a thread dump to check how many threads are in the thread pool.

Henry

[Ranganathan Kaliyur Mannar]

A FixedThreadPool will create a max of n threads at a given point. But, if you submit more tasks, once earlier threads are done and the current number of threads become less than n, then it will create further threads for those tasks.

This is what is said in the API:
"At any point, at most nThreads threads will be active processing tasks. If additional tasks are submitted when all threads are active, they will wait in the queue until a thread is available."

So, if you haven't closed the connection properly, then there is a possibility many connections being open...

[Martin Vajsar]

I'd just mention that to effectively process 20 requests in your database concurrently, the database must be sized up to the task. If it isn't, the processing might actually take longer and squeeze out other users from the database.

That said, moving the processing to the database might be much, much more effective to shorten the processing time.