• Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other Pie Elite all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Jeanne Boyarsky
  • Ron McLeod
  • Paul Clapham
  • Liutauras Vilda
Sheriffs:
  • paul wheaton
  • Rob Spoor
  • Devaka Cooray
Saloon Keepers:
  • Stephan van Hulst
  • Tim Holloway
  • Carey Brown
  • Frits Walraven
  • Tim Moores
Bartenders:
  • Mikalai Zaikin

Removing threads from my thread pool

 
Ranch Hand
Posts: 904
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Ranchers,

I've implemented a threadpool(TP) which can contain between 0 and 10
(perhabs up to 15 later) threads. Since each task/job takes 10-20
seconds to execute I use the TP to have a bound of my resource usage. The
jobs come in "flows", (i.e. 3-4 at a time) and I wish to remove the waiting
threads when they are not being used.


Option A: remove the thread from the TP when it finishes.

Option B: remove the thread when it's been inactive in 1 minut.

Which one should I choose? and what are the pros and cons?
I was thinking of using a timer and remove threads when they've been
inactive for some time. However using a timer (timertask ?) will add
a performance penalty. On the other hand, if I remove the threads
when they finish I might end in a situation where I remove a thread
only to add a new one an instance later.

Thanks in advance.

/Svend Rost
 
(instanceof Sidekick)
Posts: 8791
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
If you remove it when it ends you're not pooling at all. Let's change your choices to remove it when it ends and there are no tasks waiting, or after some idle time, or never. And let's make sure we're solving a problem. What's the worst thing that happens with each choice?
[ August 25, 2006: Message edited by: Stan James ]
 
Svend Rost
Ranch Hand
Posts: 904
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Hi Stan,

thanks for replying.

True. If the threads are removed im not polling, and I just as well
might have an integer representing the max # threads.


Stan: "make sure we're solving a problem"
Well, if more than 15 "jobs" are being processed the application will
become unstable and crash. The jobs come from "clients" which emit the jobs
from time to time. So we are solving a problem. Or? The app. doesn't have
that much CPU/memory, so any performance opmization is needed.


Choices:
A) Remove thread when it ends
Have a higher bound of the number of running threads. We dont get the
advantage that the thread doesn't have to be initialized.. on the other
hand this performance advantage might be "nothing" as it takes around
10-15 seconds to process each job. The positive thing about this solution
is that we dont have the extra overhead a TP introduces.

B) No tasks waiting
As I understand it, queing isn't an option. If a "client" doesn't get his
job processed he'll just send a new job later on.

C) After some idle time
Im not sure how much extra overhead I'll introduce if I have to implement
a timer class which will start (yet) another thread which'll remove the
thread if nothing happends within 1 minut. The possible advantage of this
approach (assuming a timer wont introduce alot of extra overhead) is that
I can make a TP which'll adapt to the work load.

D) Never remove threads
We'll have X number of live threads in the application. We'r trying to
make the application more stable, so minimizing the total number of
threads will properly be a good idea (I think). Threads are also being
used other places in the app.


Again: ideas/comments are much appriciated.

/Svend Rost
 
Stan James
(instanceof Sidekick)
Posts: 8791
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I'm not getting all of this yet. See how much of this sounds right ...

You can limit a thread pool solution to 15 threads or whatever number turns out to not destabilize the system. A firm max sounds like what you need, no?

What should happen when all are busy and another request comes in? If queuing is not an option you might start another thread - risking stability - or reject the request with an exception or error mesasge - which is my goal in another posting in the Threading forum.

So let's say we have our 15 thread limit and we've chosen between queing up or rejecting requests that arrive when all 15 are busy. Now just by statistical variations we eventually go idle with no requests in flight. My guess (untested) is the cost of leaving those 15 threads waiting for more work from the Executor's queue is very small.

It would be interesting to compare leaving them in the pool to removing them in a small experiment to see which has more overhead. Thread pooling is simply a bet that setup and teardown is worse than holding idle threads. That may or may not be true in different combinations of load and other resource demands.

Are we working in the same direction?
[ August 25, 2006: Message edited by: Stan James ]
 
Svend Rost
Ranch Hand
Posts: 904
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Yes, I need a max - but the question is if I should allow the number of
"alive" threads to shrink when there's no work.

My app reminds of a client-server solution. A central server receives requests
from up to 120 clients. A request takes around 15 secs to be processed and
if the server doesn't send a request back the client will send another
request later on. Sometimes the clients send requests simultanious which
is why I need a max number of active threads.

My guess (untested) is the cost of leaving those 15 threads waiting for more work from the Executor's queue is very small


Thread pooling is simply a bet that setup and teardown is worse than holding idle threads. That may or may not be true in different combinations of load and other resource demands.



Okay thanks.. My initial question was what was the best, i.e. gave
the best performance: fixed num of threads, pool with max num of threads or
an pool which removed the threads after an perod of inactivity.

When there's alot of work to be done, i.e. lots of requests from the clients
the question is: What introduces most overhead,
A) Starting new threads
B) Using a timer in the TP
C) Reusing Threads

As I mentioned earlier: A and B has the advantage, that when there's no
work to do there's no extra overhead by having 15 alive threads. Using
B however have the disadvantage that I assume it must introduce extra
overhead by having to start a new timer.

Please correct me if im wrong. How should I go about implementing the
timer btw?`one per thread, or one per pool?

/Svend Rost
 
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I don't see where you have established that there is any good reason to trim currently unused Threads out of the pool. Surely the overhead is minute compared to the cost of creating a new Thread.

IF you decide that there is a good reason to trim, there is still no reason to check at any frequent interval.
How about this - a timer wakes up at one minute intervals and looks at the count of waiting Threads - if the count is > some parameter, remove N, if the count is less than some parameter, add N.

Bill
 
Svend Rost
Ranch Hand
Posts: 904
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
Thanks for joining the conversation Bill. Your feedback is appriciated.

Originally posted by William Brogden:
I don't see where you have established that there is any good reason to trim currently unused Threads out of the pool. Surely the overhead is minute compared to the cost of creating a new Thread.


My reason for removing the unused threads is that I assume the threads
add a performance penalty. I might be wrong though - perhabs 15 threads
doesn't use that many ressources. As I have explained earlier, the application
isn't running that stable and threads are being used elsewhere in the app.
so I figured that the more alive threads I could remove, the better.

How about this - a timer wakes up at one minute intervals and looks at the count of waiting Threads...


Thanks - sounds like a plan. I'll definitly consider it it I decide to
trim the pool size.
 
Stan James
(instanceof Sidekick)
Posts: 8791
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
I mentioned that gamble above ... I'd bet that creating and destroying threads contributes more to stability problems than leaving a pool sitting idle. We know for sure that profile fit enough servers to make the invention of thread pooling worth while. You've mentioned a couple times that the app uses threads elsewhere. It might even be worth looking to see if pooling would work in those other places, too.

Editing now - you mentioned clients that retry, too. Do you control that code? You might want to limit a client to one or two retries, maybe spread them out a bit. We had to tune client retries on one system to avoid making things worse instead of better.
[ August 27, 2006: Message edited by: Stan James ]
 
Svend Rost
Ranch Hand
Posts: 904
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by Stan James:
You've mentioned a couple times that the app uses threads elsewhere. It might even be worth looking to see if pooling would work in those other places, too.


Yes, good idea. It's also what I've done. The requests are
processed rather quickly, and the requests come in great numbers, so im
pretty sure the pool gives me a performance bonus.

Originally posted by Stan James:

Editing now - you mentioned clients that retry, too. Do you control that code?


Sadly no. The clients are hardware units which we have no control over.


I'd bet that creating and destroying threads contributes more to stability problems than leaving a pool sitting idle.


With some luck I can get a profiler to work, which will provide me with
some quantitative data about what to choose.

Thanks for the advice.
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

I assume the threads add a performance penalty. I might be wrong though - perhabs 15 threads doesn't use that many ressources.



My Tomcat installation has a typical configuration where the thread pool for requests has max 150, min spare threads = 25, max spare threads = 75
so you can see that the Tomcat designers don't think having lots of spare threads hanging around is a big problem. This system runs for weeks with only 25 threads actually allocated in the pool.

A Thread gets a memory allocation when created but that does NOT mean that an idle one is impacting memory use since the operating system can page that memory out. For example, right now the operating system task manager shows actual memory used by my Tomcat is 41,528k but the tomcat manager shows that the JVM has 70mb of which 15 is unused at the moment.

Bill
 
Svend Rost
Ranch Hand
Posts: 904
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by William Brogden:

My Tomcat installation has a typical configuration where the thread pool for requests has max 150, min spare threads = 25, max spare threads = 75
so you can see that the Tomcat designers don't think having lots of spare threads hanging around is a big problem. This system runs for weeks with only 25 threads actually allocated in the pool.


This app isn't a web app - I have much less resources. This app has to run
on 128 MB all inclusive (linux OS, JVM ect). If more than 10 "big processes"
are being processed concurrently, the app starts to become unstable.


Originally posted by William Brogden:

A Thread gets a memory allocation when created but that does NOT mean that an idle one is impacting memory use since the operating system can page that memory out.


Okay - thanks.
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

This app has to run on 128 MB all inclusive (linux OS, JVM ect).



Aha - that certainly puts a different slant on things, a real challenge.
What sort of functionality do you have in the Thread pool so far?

Using 10-20 seconds per job sounds like serious computation is going on - how memory intensive is each job? Having extra Threads started may slow things down if the operating system has to thrash pages in and out.
 
Wanderer
Posts: 18671
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
[Stan]: If you remove it when it ends you're not pooling at all.

True, but from what I've seen here, I'm not sure full pool functionality is necessary here. My sense is that the ten threads themselves are not that big a problem, but the additional resources they use while performing a task are significant. So what I'm thinking is, maybe there's no need to save these threads at all. Just keep track of how many are currently running, and do something like this:

So every new task gets a new thread rather than using an existing thread - but you never have to keep the threads around longer than needed, either.

To keep count of how many active threads there are, and detect when the number is low enough to start a new thread, you could create a custom Thread or Runnable which wraps the task, incrementing a counter when it starts, and decrementing it on exit, with a notify() a the end to let a waiting thread know there's a change in status. Or just use ThreadGroup.activeCount() to get an approximate current count (probably good enough) and use a polling loop to periodically check for changes. The first strategy seems more refined to me, but the latter may be good enough as a quick & dirty approach. Either of these seems simpler to me than a full thread pool, and probably good enough for the problem described.
 
Svend Rost
Ranch Hand
Posts: 904
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator

Originally posted by William Brogden:

What sort of functionality do you have in the Thread pool so far?


addJob(Runnable) - called when someone wishes to get some work done.
if "available thread" => notify()
if "still room in the pool but no available thread" => new Worker Thread is started

getNextJob() - called by a Worker Thread
if the list of jobs is empty =>
enter loop: you get out of the loop (in which you wait() abit) when
a) free threads > min pool size -> return null
b) the list of job isn't empty -> exit loop

if the list of jobs isn't empty => return a job from the list

A worker thread can only exit "his" run method when the job he gets
is null.

Originally posted by William Brogden:

Using 10-20 seconds per job sounds like serious computation is going on - how memory intensive is each job? Having extra Threads started may slow things down if the operating system has to thrash pages in and out.


Im not quite sure how memory intensive it is. I can not run any profilers
as im using a scaled down version of J2SE. I'll try to investigate it.
There are alot of things that's being done: identifying client, validating,
updating information about the client, replying back to the client.
 
William Brogden
Author and all-around good cowpoke
Posts: 13078
6
  • Mark post as helpful
  • send pies
    Number of slices to send:
    Optional 'thank-you' note:
  • Quote
  • Report post to moderator
At 10 seconds per job, the resources used to create a Thread are minute compared to the rest of the job. I think Jim is on the right track - just keep track of the number of jobs running so you can refuse to accept too many. Can you put surplus requests in some sort of "waiting for resources" loop that will let the client know whats happening?

Bill
 
With a little knowledge, a cast iron skillet is non-stick and lasts a lifetime.
reply
    Bookmark Topic Watch Topic
  • New Topic