aspose file tools*
The moose likes Threads and Synchronization and the fly likes Thread pool size Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "Thread pool size" Watch "Thread pool size" New topic
Author

Thread pool size

Kedar Thakar
Greenhorn

Joined: Jan 04, 2012
Posts: 4
For one of the my project I want to change the thread pool size runtime. I have taken Executors.newfixedthreadpool(X) to create the thread pool. After gap of 10/15 seconds I am checking my whole task will get finished within stipulated time or not. If task is running behind scheduled time, then I am increasing the size using Executors.newfixedthreadpool(X+Y).

Is this right way of increasing the thread pool size or there is alternate available to increase the size. Please guide me on this.

Thanks in advance.
Ireneusz Kordal
Ranch Hand

Joined: Jun 21, 2008
Posts: 423
Hello,

consider ThreadPoolExecutor --> webpage
Actually Executors.newFixedThreadPool is the ThreadPoolExecutor, but with corePoolSize = maximumPoolSize, look at the source code (lines 88-92)--> webpage


In the ThreadPoolExecutor you can separately set a corePoolSize to X and a maximumPoolSize to X + Y,
and this pool automatically creates additional threads when needed (if the pool queue is full) - up to maximumPoolSizie,
and, on the other hand, if some threads are idle for more than keepAliveTime time, these unnecessary threads are destroyed and removed from the memory,
keepeng active only corePoolSize threads.
Kedar Thakar
Greenhorn

Joined: Jan 04, 2012
Posts: 4
Thanks for the info. But this will again hardcoding of the thread size. I will explain the scenario with example.
Actually I am starting with thread size as 1.

Executors executor = null;

method ABC(){
executor = Executors.newfixedthreadpool(1);
for(){
executor.submit(callable);
checkTime();
}
}
//Method checkTime
method checkTime(){
if(time < expectedtime){
//DO some calculation and find out the number of threads i.e. Y
executor = Executors.newfixedthreadpool(Y);
}

}

checkTime method will run for around 20-25 time to check the time schedule.
Reinitialization of executor in the checkTime method to increase the fixed thread pool is correct way? Or is there any alternate available.

Thanks.
Alexander Kober
Ranch Hand

Joined: Aug 05, 2011
Posts: 32

At Th wrote:Thanks for the info. But this will again hardcoding of the thread size.


Actually, the thread pool described in the answer above will adapt dynamically to the number of jobs, it will just do so based on the number of queued tasks rather than on a fixed interval.

However, let's take one step backwards first. What kind of jobs are you processing? Knowing this is critical, as there are few cases where just increasing the number of threads will actually give you more performance (meaning both latency and throughput), and even then you can't just infinitely scale the number of threads. If you don't know what tasks are being processed, you don't have a solid basis for estimating the optimal number of threads anyway. If you know what is being done, you should be able to find the optimal maximum number of threads based on the number of CPUs in the system (Runtime.getRuntime().availableProcessors()).

Kedar Thakar
Greenhorn

Joined: Jan 04, 2012
Posts: 4
Thanks for the reply. In my case I dont know the exact task and its number. Task can be 100 in number or may grow till lacs. In that case what should be my approach. Also, I woluld like to know the approach I took to change the thread size is correct or not?

Thanks
Alexander Kober
Ranch Hand

Joined: Aug 05, 2011
Posts: 32

Kedar Thakar wrote:Thanks for the reply. In my case I dont know the exact task and its number. Task can be 100 in number or may grow till lacs. In that case what should be my approach. Also, I woluld like to know the approach I took to change the thread size is correct or not?

Thanks


I have a feeling you might be making a mistake here distinguishing between threads and tasks. Tasks, as in work packages, may be unlimited. The number of threads only determines how many tasks are being worked on at the same time (and, implicitly, how many CPU cores are being used). It is almost generally a bad idea to design a system to have hundreds of active threads, with the exception of few heavy duty server applications. Having significantly more threads than CPU cores will have a severe impact on throughput. The question at hand is: Do you really need the tasks to be run at the same time, or can't you just process them one by one using a smaller number of threads? Or, to be more precise: Will your tasks time out if they're not handled immediately?

Regarding the correctness of your approach: I would not recommend simply 'throwing away' the old executor the way you are now. Also, your method checkTime, at least judging from your pseudo-code, simply measures the time it takes to enqueue the task for the pool. This does not in any way mean that the task is completed when executor.submit(...) returns, only that the executor accepted the task as something to be done at some point.
 
Consider Paul's rocket mass heater.
 
subject: Thread pool size