Design decisions around using ExecutorService

[Sven Anderson]

Hi,

I'm a bit puzzled about some of the concepts around threading and in particular design decisions around using ExecutorService classes.

I'm currently implementing a multi-threaded apps using JDK1.5 and Spring. I have build a Service class which will do a number of IO intense (DB and Web service) operations in order to create a composite object which I return to the user. My first attempt was to create and instantiate an ExecutorService object for each method invocation on this service class (see example1) and then perform a shutdown at the end of the method. Thinking about it, since there will be at least 1 call per second to this service class. To me it sounds like plenty of overhead having to create an ExecutorService and then tear it down for each request. Wouldn't it be better to create an ExecutorService threadpool at application startup (wired in Spring xml) and inject this executorService into each service class that needs threading support? Then the threadpool will only be created once and all the threads in the pool will be shared across the various Service classes that needs threading support. In this case, when do I perform a shutdown? Only during restart of the application? Also, having threads waiting for incoming request, is this what I want, or would I be better of closing the pool and reopen it again?

Bit confused here

Thanks
Sven

Example 1:

public class ServiceAImpl() implements MyServiceA {
 
// setters for dao's
 
public MyCompositeObject getQuote(Query query){
MyCompositeObject o;
 
// I'll be calling 10 different datasources
ExecutorService executor = Executors.newFixedThreadPool(10);
 
executor.submit(xzy);
// followed by submit of other Runnable or Callable classes
 
// code to populate MyCompositeObject
 
executor.shutdown();
return o;
}
    

Example 2:

public class ServiceAImpl() implements MyServiceA {
 
// setters for dao's
private ExecutorService executor;
public void setExecutor(ExecutorService ex){
this.executor = ex;
}
 
public MyCompositeObject getQuote(Query query){
MyCompositeObject o;
 
// I'll be calling 10 different datasources
executor.submit(xzy);
// followed by submit of other Runnable or Callable classes
 
// code to populate MyCompositeObject
 
executor.shutdown();
return o;
}
 
    

[Nitesh Kant]

Probably, having a single thread pool for all the services will not be a great idea. The primary reason being that you will not be able to prioritize a service over others i.e. if you have a service that has more and important tasks and another one that has less but time consuming tasks. Then, it may so happen that the time consuming tasks are ahead in the queue and the thread pool threads are busy processing them.
However, the important tasks are behind in the queue and does not get processed fast enough.
Of course, if you do not have such a requirement then you might as well use a single executor.

Starting and stopping executor for a single request is not at all a good idea.

Typically, most of the applications start the executors at startup and shutdown at the application stop(if you can hook on to such events) or a user action.

Injecting the executor in the service class is a good idea.
Alternatively, you can create a registry of executors that the service class can query.
Of course, this will bring a bit of tight coupling between service class and the registry.

[Sven Anderson]

Thanks for a well explained answer,

You raised a very interesting point about using several thread pools in order to prioritise task executions.In my case I'm planning to take advantage of threadings for the following components in my app.

* A dispatcher which serves incoming client requests.
* ServiceA which will concurrently go and fetch data from 10 datasources
* ServiceB which will fetch data based on n number of results returned from a query


I been given the advice to not create threadpools that are greater than the number of server processors (I think this was also mentioned in the book Java Concurrency in Practice). Given I've only got an 8 core server. I guess I need to configure the threadpools wisely. I'm not really sure about the different factors to consider when configuring the different pool sizes. Obviously I want to dedicate as many threads as possible to the dispatcher but on the other hand I would also like to give ServiceA or B as many threads as possible. To me it sound a little bit like I'm moving the possible bottlenecks between the dispatcher and the Service classes.

[Nitesh Kant]

Sven:
I been given the advice to not create threadpools that are greater than the number of server processors.

I believe, you meant number of *threads* must not be greater than the number of processors.
Yeah, that is correct but in the case where your processing is 100% CPU bound i.e. it does not stop a fraction of a nanosecond for anything but just keeps on processing
However, in most of the real life scenarios, this is not the case.
Moreover, in your case it does not look like that at all as you are querying a database which is I/O bound.
So, the best way is to profile your application and find out how is it performing at high loads. If you see that you have idle CPUs then you can tweak the number of threads and try out.
Typically, there is no magic rule that will tell you what is the optimal number of threads. You have to profile and find out yourself.

Sven: I'm not really sure about the different factors to consider when configuring the different pool sizes.

I would like to give the dispatcher more number of threads since client response time is always a priority.