wood burning stoves 2.0*
The moose likes Performance and the fly likes Capacity planning (performance tuning of WLP 8.1 SP6 application) Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Java » Performance
Bookmark "Capacity planning (performance tuning of WLP 8.1 SP6 application)" Watch "Capacity planning (performance tuning of WLP 8.1 SP6 application)" New topic
Author

Capacity planning (performance tuning of WLP 8.1 SP6 application)

Jameel Khan
Greenhorn

Joined: Aug 07, 2007
Posts: 1
Hi Guys,

I have one question regarding the performance tuning of the weblogic portal application (or may be any J2EE application for that matter) :

I understand that there could be lot of performance bottleneck areas e.g
(a) network issues (not enough bandwidth),
(b) bottleneck because of the DBs,
(c) application design/coding
(d) JVM might not have been tuned as per the business requirement of the application
(e) lack of clustering
(f) hardware problems (CPUs, RAMs etc)

Inorder to do the capacity planning for such an enterprise applications, what would be the best starting point?

Is it a good idea to start and profile the application using some profiling tools like OptimizeIt or JProbe? and then analyze the profiling the data and conclusion can be drawn interms of what and where the major bottleneck is into the application and accordingly it can be worked upon.

Or if there is any better and elegant way doing capacity planning?

Thanks for your time.

Regards
Jameel
Rejeev Divakaran
Greenhorn

Joined: Aug 12, 2007
Posts: 27
I think Performance Tuning and Capacity planning are different things.
Capacity planning - decide the hardware and software configuration required for specific application load (specific TPS)
Performance Tuning - Tune the application/software/hardware to optimize the TPS for a specific hardware/software configuration.
Typically we will be doing the performance tuning as part of application development. Capacity planning before deploying the application for any customer. However we may need to do further performance tuning after capacity planning.
Performance tuning:
First pass:
1) Do a load testing for 15 minutes.
2) Monitor the different parameters
a) CPU
b) Time spent in application vs time spend in Database (or any other external system)
c) Garbage collection (time spend in GC vs total time)
d) Take multiple thread dump and look for threads waiting for monitor entry (you may use samurai thread dump analysis)
e) network i/o rate
f) if total commit charge exceeds the physical memory
Analysis:
if more time spend on DB tune DB
if lot of threads in waiting for monitor entry, tune code to remove synchronization blocs
if lot of time spend on GC, try increase/decrease heap, multiple JVMs
if total commit charge exceeds physical memory decrease heap size
if network i/o is close to i/o capacity, use compression to reduce i/o
Second pass:
Run the application with CPU profiler
Find the hotspots. Analyze the code for optimization.
Capacity planning:
Run load testing for 15 minutes with application on standard machine and database other dependencies in higher machines
measure the TPS
Find TPS per CPU
network i/o per transaction (typically this may not be a big issue)
Calculate the number of CPUs required
Decide number of machines (m number of n CPU machines)
Load testing:
1) Find the key transaction (end user input)
2) Find the ratio of key transactions (end user input)
3) Write load testing stubs (using some load testing tool)
4) Increase the number of threads until you reach CPU usage of application to 80+%
DB tuning (oracle 9.2)
take snaps before and after load testing
take statspack report
look at top 5 timed events.
Tune DB to reduce item which is taking more time. Typically increase db buffer size, increase pga aggregate etc
Find the Sqls taking more time.
Find the explain plan for these SQLs
Tune these SQLs to have better plan. Typically proper indexing will do.


Regards,<br />Rejeev<br />Blog: <a href="http://rejeev.blogspot.com/" target="_blank" rel="nofollow">http://rejeev.blogspot.com/</a>
Tony McClay
Ranch Hand

Joined: May 22, 2003
Posts: 33
WOW! Rejeev Divakaran said it better than I think I have ever read it before, in such a great summarized form.

I will offer a few things..
1. Find an Expert
If this is mission critical, it may be worth it to find someone who has had experience in performance tuning with your environment/servers. Relationships with local consulting companies are great resources. They want to get in and be able to help you, and usually the more one does this type of work, the better they are at it.

2. The Top 5
Like a letterman show.. usually the Top 5 performance bottlenecks may be responsible for 90% of your performance issues. We don't have JProbe, or Optimize it, but there are some good opens source/free-ish tools, with Netbeans, and Eclipse, which is a good start if you have nothing.

3. Allow Time
All too often optimization, performance tunning, capacity planning are done at the end of project life cycle. Try to integrate them into the development cycle. Ant/Cruise control and a number of open source projects has the means to assist you in this effort. Surprises at the end of a project are never kind. Especially the ones that require capital and resources. Those don't usually come quickly as well.

4. Be cautious of publicized capacities
Remember, application servers have a goal of outdoing each other. Yes there are some independent benchmarks but you must test your application before you have a true graps on the capacity of your application/infrastructures. I have had to come up is scales of formulas many times, because I have been forced to come up with numbers/resources, even before the project is completed and maybe not the same hardware, but this is always a key point.

5. Underestimate / (or Over estimate - however you think of it)
Everyone loves being surprised to handle more requests, faster than expected. No one is happy with a new system does less. That's a bad feeling having to either further optimize or explain why.

I like Rejeev Divakaran, post better than mine.

Tony
Sun Certified Web Business Component Developer
Sun Certified Web Components Developer
Sun Certified Programmer for the Java 2 Platform


Tony McClay<br />Architect / Developer, SOA and Jave Enterprise Edition 1-5<br />---------------------------------------------------------------------- <br />Sun Certified Enterprise Architect, Enterprise Edition 5 (Step 1 of 3)<br />Sun Certified Web Component Developer, Enterprise Edition 4<br />Sun Certified Business Component Developer Enterprise Edition 5<br />Sun Certified Programmer , Standard Edition 5.0
Rejeev Divakaran
Greenhorn

Joined: Aug 12, 2007
Posts: 27
Thanks Tony for the compliment.
I have recently tried YourKit profiler and I found it very useful. It is much less expensive than JProbe and OptimizeIt (By the way OptimizeIt is retired).
It doesn't support method tracing with JDK 1.4.x Anybody knows any tool (referably free)supports method tracing with JDK 1.4?
 
jQuery in Action, 2nd edition
 
subject: Capacity planning (performance tuning of WLP 8.1 SP6 application)