File APIs for Java Developers
Manipulate DOC, XLS, PPT, PDF and many others from your application.
http://aspose.com/file-tools
The moose likes Threads and Synchronization and the fly likes Optimization of Thread Pool Executor Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of OCM Java EE 6 Enterprise Architect Exam Guide this week in the OCMJEA forum!
JavaRanch » Java Forums » Java » Threads and Synchronization
Bookmark "Optimization of Thread Pool Executor" Watch "Optimization of Thread Pool Executor" New topic
Author

Optimization of Thread Pool Executor

Tushar Java Dev
Greenhorn

Joined: Oct 04, 2013
Posts: 2
Hi,

I am using ThreadPoolexecutor by replacing it with legacy Thread.

I have created executor as below:

pool = new ThreadPoolExecutor(coreSize, size, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(coreSize), new CustomThreadFactory(name),new CustomRejectionExecutionHandler());
pool.prestartAllCoreThreads();

here core size is maxpoolsize/5
and i have prestarted all the core threads on start up of application roughly around 160 threads.

in legacy design we were creating and starting around 670 threads.

But the point is even after using Executor and creating and replacing legacy design we are not getting much better results.

For results memory management we are using top command to see memory usage


and for time we have placed loggers of System.currentTime in millis to check the usage.

Please tell how to optimize this design

Thanks & Regards,
tushar
Steve Luke
Bartender

Joined: Jan 28, 2003
Posts: 4181
    
  21

Tushar Java Dev wrote:Please tell how to optimize this design

Don't do it by guessing. Get a profiler, run the application with the Profiler attached. Find out where the code bottlenecks are. Address the bottlenecks. Nothing else will be a functional approach to optimizing the design. If you just guess (i.e. don't measure with a profiler) then you are as likely to make bad choices as good.


Steve
Tushar Java Dev
Greenhorn

Joined: Oct 04, 2013
Posts: 2
Steve Luke wrote:
Tushar Java Dev wrote:Please tell how to optimize this design

Don't do it by guessing. Get a profiler, run the application with the Profiler attached. Find out where the code bottlenecks are. Address the bottlenecks. Nothing else will be a functional approach to optimizing the design. If you just guess (i.e. don't measure with a profiler) then you are as likely to make bad choices as good.



Please elaborate which profiler to use and how to use it.
Steve Luke
Bartender

Joined: Jan 28, 2003
Posts: 4181
    
  21

There are multiple profilers available. Try searching for one. Using one is usually pretty simple: some variation of You launch the profiler, you launch the Java application, you select the Java application, you collect data. Making use of one, on the other hand, is a bit more difficult to explain and best learned through the particular software's help system.
Jayesh A Lalwani
Bartender

Joined: Jan 17, 2008
Posts: 2343
    
  28

What made you think that going from threads to a thread pool will make your application faster? The advantage that a thread pool gives is that you don't need to start and shut down a thread every time you run a background tasks. Creation of threads is costly, so it keeps threads running in the background. You will se performance gain only when you are continiously running tasks all day. If all you are doing is starting 670 threads, executing them and then shutting down and not using them ever again, you won't see any performance gain from using a thread pool

You have 900 threads now? What made you think that starting a shit load of threads is going to improve performance? Is this like a mainframe box you are running on? Your performance is always going to be constrained by the size of your hardware. Threads are not magic devices that help you break the laws of physics. Any task that you run is going to take some CPU, some memory, might incur some IO locally, might incur some IO over the network, and might use up resources on some external server (like a database or something). If your tasks use more CPU, they are CPU bound.. more memory, then they are memory bound.. more IO, they are IO bound Adding threads to your application is going to help you make better use of your resources, but it's not going to magically add more CPU, memory or IO into your machine. So, let's say your tasks are CPU bound and each task takes 10% of one CPU, and you have 4 CPUs. That means you cannot have more than 40 going on at the same time. Actually, you shouldn;t be having 40 going on at the same time. You should be just under it. Maybe 35. The CPU is physically going to be busy switchign between threads if you cross the threshold that causes your CPU usage to go to 100%, you are going to be busted. Your performance will actually reduce because CPU will be doing a lot of context switching. The same thing with memory. If you have 2GB of memory, and each thread takes 256MB, you are going to be full at 8 threads. You add more than 8 threads, and GC is going to go crazy.

So, yeah, what you need to do is start with 1 thread and ramp it up and keep an eye on CPU, memory and IO usage (most probably you don;t need to be worried about IO.. if you have a 10G pipe you are probably going to be running out of CPU memory long before). Once you hit the limit you gotta stop increasing the number of threads. You physically don;t have more resources on the system. You are out of juice. Increasing your threads is going to slow down your throughput at this point.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Optimization of Thread Pool Executor