Win a copy of Re-engineering Legacy Software this week in the Refactoring forum
or Docker in Action in the Agile forum!
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic

Thread pool size

 
Kedar Thakar
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
For one of the my project I want to change the thread pool size runtime. I have taken Executors.newfixedthreadpool(X) to create the thread pool. After gap of 10/15 seconds I am checking my whole task will get finished within stipulated time or not. If task is running behind scheduled time, then I am increasing the size using Executors.newfixedthreadpool(X+Y).

Is this right way of increasing the thread pool size or there is alternate available to increase the size. Please guide me on this.

Thanks in advance.
 
Ireneusz Kordal
Ranch Hand
Posts: 423
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Hello,

consider ThreadPoolExecutor --> webpage
Actually Executors.newFixedThreadPool is the ThreadPoolExecutor, but with corePoolSize = maximumPoolSize, look at the source code (lines 88-92)--> webpage


In the ThreadPoolExecutor you can separately set a corePoolSize to X and a maximumPoolSize to X + Y,
and this pool automatically creates additional threads when needed (if the pool queue is full) - up to maximumPoolSizie,
and, on the other hand, if some threads are idle for more than keepAliveTime time, these unnecessary threads are destroyed and removed from the memory,
keepeng active only corePoolSize threads.
 
Kedar Thakar
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the info. But this will again hardcoding of the thread size. I will explain the scenario with example.
Actually I am starting with thread size as 1.

Executors executor = null;

method ABC(){
executor = Executors.newfixedthreadpool(1);
for(){
executor.submit(callable);
checkTime();
}
}
//Method checkTime
method checkTime(){
if(time < expectedtime){
//DO some calculation and find out the number of threads i.e. Y
executor = Executors.newfixedthreadpool(Y);
}

}

checkTime method will run for around 20-25 time to check the time schedule.
Reinitialization of executor in the checkTime method to increase the fixed thread pool is correct way? Or is there any alternate available.

Thanks.
 
Alexander Kober
Ranch Hand
Posts: 32
IntelliJ IDE Java Scala
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
At Th wrote:Thanks for the info. But this will again hardcoding of the thread size.


Actually, the thread pool described in the answer above will adapt dynamically to the number of jobs, it will just do so based on the number of queued tasks rather than on a fixed interval.

However, let's take one step backwards first. What kind of jobs are you processing? Knowing this is critical, as there are few cases where just increasing the number of threads will actually give you more performance (meaning both latency and throughput), and even then you can't just infinitely scale the number of threads. If you don't know what tasks are being processed, you don't have a solid basis for estimating the optimal number of threads anyway. If you know what is being done, you should be able to find the optimal maximum number of threads based on the number of CPUs in the system (Runtime.getRuntime().availableProcessors()).

 
Kedar Thakar
Greenhorn
Posts: 4
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thanks for the reply. In my case I dont know the exact task and its number. Task can be 100 in number or may grow till lacs. In that case what should be my approach. Also, I woluld like to know the approach I took to change the thread size is correct or not?

Thanks
 
Alexander Kober
Ranch Hand
Posts: 32
IntelliJ IDE Java Scala
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Kedar Thakar wrote:Thanks for the reply. In my case I dont know the exact task and its number. Task can be 100 in number or may grow till lacs. In that case what should be my approach. Also, I woluld like to know the approach I took to change the thread size is correct or not?

Thanks


I have a feeling you might be making a mistake here distinguishing between threads and tasks. Tasks, as in work packages, may be unlimited. The number of threads only determines how many tasks are being worked on at the same time (and, implicitly, how many CPU cores are being used). It is almost generally a bad idea to design a system to have hundreds of active threads, with the exception of few heavy duty server applications. Having significantly more threads than CPU cores will have a severe impact on throughput. The question at hand is: Do you really need the tasks to be run at the same time, or can't you just process them one by one using a smaller number of threads? Or, to be more precise: Will your tasks time out if they're not handled immediately?

Regarding the correctness of your approach: I would not recommend simply 'throwing away' the old executor the way you are now. Also, your method checkTime, at least judging from your pseudo-code, simply measures the time it takes to enqueue the task for the pool. This does not in any way mean that the task is completed when executor.submit(...) returns, only that the executor accepted the task as something to be done at some point.
 
  • Post Reply
  • Bookmark Topic Watch Topic
  • New Topic