File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Threads and Synchronization and the fly likes ThreadPool, don't execute two jobs simultaneously if certain conditions are met Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Android Security Essentials Live Lessons this week in the Android forum!
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "ThreadPool, don Watch "ThreadPool, don New topic
Author

ThreadPool, don't execute two jobs simultaneously if certain conditions are met

Sebastian Janisch
Ranch Hand

Joined: Feb 23, 2009
Posts: 1183
hey guys,

let's say we create a thread pool with a pool size of 2. Now we add 4 jobs, named job1, ..., job4.

is there a way to tell the thread pool that job1 and job2 must not be executed simultaneously, but get added into a queue?


Thanks

Sebastian


JDBCSupport - An easy to use, light-weight JDBC framework -
Stephan van Hulst
Bartender

Joined: Sep 20, 2010
Posts: 3600
    
  14

Add one job to the pool that handles both jobs 1 and 2, one after the other.

This smells a bit fishy though. Why do you want to do this?
Sebastian Janisch
Ranch Hand

Joined: Feb 23, 2009
Posts: 1183
hey,

thanks for your reply ...

why do we want to do this ...

imagine there are 200 sources that pump in tasks ... the requirement is that tasks that come from the same source must not get executed at the same time, tasks from different sources can, though ...

so there should be one threadpool with, say, 40 threads that those tasks from 200 sources get pumped in to but it needs to watch out not to execute two tasks from the same source at once.

we could create one threadpool for each of those 200 sources, but that doesn't make any sense as 200 * 40 = a lot of threads ...
Stephan van Hulst
Bartender

Joined: Sep 20, 2010
Posts: 3600
    
  14

No, you would create one thread per source, because each source should be handled by only one thread anyway. If this is not acceptable, and you want to use a fixed pool regardless of how many sources you have, then you can just make your own construction using the ExecutorService.submit() method. I don't readily know if there's a better option available, but this shouldn't be too hard to implement yourself.

It depends on how you get notified of a new job though. How do you retrieve jobs from sources?
Sebastian Janisch
Ranch Hand

Joined: Feb 23, 2009
Posts: 1183
hey,

it's a client server app and tasks come in from various sources ... and it's important that tasks from the same source are processed synchronously ...
Stephan van Hulst
Bartender

Joined: Sep 20, 2010
Posts: 3600
    
  14

No what I mean is, how do get the jobs, on a more concrete level? Do you use a ServerSocket and block until something comes in? Do you actively retrieve jobs or do you passively wait until you get notified of one, etc?
Sebastian Janisch
Ranch Hand

Joined: Feb 23, 2009
Posts: 1183
the whole thing uses the jboss netty server socket framework and as messages come in, tasks are created that belong to a certain group ...
Stephan van Hulst
Bartender

Joined: Sep 20, 2010
Posts: 3600
    
  14

I guess what you can do is make some sort of multiplexer class.

Make a queue for every source you have and register it with your multiplexer. Now what the multiplexer does is it polls every queue associated with a source that had its last task handled, to see if it has a new task. If it does, then it simply submits that task to your thread pool.

If you give me a minute, I'll have an example for you.
Sebastian Janisch
Ranch Hand

Joined: Feb 23, 2009
Posts: 1183
hey,

I have created such class, but i'm facing synchronization issues ... here's the code... it's pretty long but most of it are comments. .. would be great if you could help finding the error ..




Nitesh Kant
Bartender

Joined: Feb 25, 2007
Posts: 1638

I feel it is better to put the throttling logic in the queue implementation itself. The reason being that the implementation is cleaner than maintaining the buffer in the executor.

The following is a way to implemented such a throttled queue:


This depends on the following callback interface:


and the following task implementation (Although you can use a different task definition as per your convenience):



In order to use the above queue, it is mandatory to use a thread pool executor where the core pool size and the max pool size is the same and all the core threads a re pre-started. The reason for this is that the executor does not enqueue it to the queue if there are spare threads that can be created. The following will be one helper class to do that:



I have just created a sample loader class to test the above:









apigee, a better way to API!
Stephan van Hulst
Bartender

Joined: Sep 20, 2010
Posts: 3600
    
  14

Here is a way you could do it from the other end, by having one dedicated thread pumping the tasks into the thread pool. I have not tested it, I don't know if it works, but it should be an interesting alternative.

Nitesh' way is much nicer though, having a custom BlockingQueue. I just thought it was nice to see if things worked on the other end.



The idea is that you keep a JobQueue for every source, and register them with a JobMultiplexer. The multiplexer will submit jobs to a thread pool. You need one dedicated thread running the process method in order to make it work. This is why Nitesh' solution is nicer.
Stephan van Hulst
Bartender

Joined: Sep 20, 2010
Posts: 3600
    
  14

Actually, after a simple test, my solution is almost sure to deadlock because the processing thread doesn't get notified when jobs are completed.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: ThreadPool, don't execute two jobs simultaneously if certain conditions are met
 
Similar Threads
Thread --Extend vs implements
Java Thread vs JMS Queue
Thread related
Multiple Thread Pool Framework - HowTo
Jenkins - Each job 'Workspace' folder structure differ from windows 7 to Windows XP.