aspose file tools*
The moose likes Performance and the fly likes tech journalist seeking comments of post-deployment performance tuning Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of EJB 3 in Action this week in the EJB and other Java EE Technologies forum!
JavaRanch » Java Forums » Java » Performance
Bookmark "tech journalist seeking comments of post-deployment performance tuning" Watch "tech journalist seeking comments of post-deployment performance tuning" New topic
Author

tech journalist seeking comments of post-deployment performance tuning

Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
I've not visited JavaRanch before so if this kind of post isn't appropriate, please holler and I'll go elsewhere. I've taken over for Esther Schindler (who pointed me to JavaRanch) as the Best Practices columnist for Software Test & Performance Magazine (http://stpmag.com). Presently, I'm trying to get a handle on the topic for my next column -- post-deployment performance tuning -- due to my editor mid-April. Would it be ok to ask a few questions on the topic here? Thanks. And if not, sorry for the intrusion!


--------------------<br />Geoff Koch<br />Freelance technical writer, journalist<br />Lansing, MI<br />AIM: geoffinmichigan<br />Yahoo: geoff_pdx<br />MSN: geoffarnoldkoch
Reid M. Pinchback
Ranch Hand

Joined: Jan 25, 2002
Posts: 775
Given the name of the magazine, FYI there are two forums here that might be of use to you, this one for performance, another one for testing. Re: questions, journalists are people too so I don't see why you shouldn't have the same opportunities as anybody else around here.


Reid - SCJP2 (April 2002)
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 852
The postings by esther generated useful responses for all developers, so I agree with the previous poster (except I am not sure if journalist really qualify as people).


http://www.jamonapi.com/ - a fast, free open source performance tuning api.
JavaRanch Performance FAQ
William Brogden
Author and all-around good cowpoke
Rancher

Joined: Mar 22, 2000
Posts: 12675
    
    5
What a huge topic - the field needs to be subdivided because Java is used for so many things - you might want to post specific queries in several forums:
Servlets and JSP forums for web applications
EJB etc for J2EE based web applications
JDBC for database tuning - (frequently the cause of performance problems in all other topics )
Java 2 Micro Edition for phone etc applications
.. and probably some others

Bill


Java Resources at www.wbrogden.com
Jeanne Boyarsky
internet detective
Marshal

Joined: May 26, 2003
Posts: 29220
    
135

Geoff,
Welcome to JavaRanch!

Esther has used JavaRanch for this purpose - it is certainly ok. She generally gets good responses when she asks some leading/thought provoking questions. Take a look at this one.


[Blog] [JavaRanch FAQ] [How To Ask Questions The Smart Way] [Book Promos]
Blogging on Certs: SCEA Part 1, Part 2 & 3, Core Spring 3, OCAJP, OCPJP beta, TOGAF part 1 and part 2
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
(Ack. I'm truly a greenhorn. I posted this by mistake as a new thread instead of as a reply. Sorry about that. Can someone delete the new thread? Or maybe I can...)

Thanks, all. I appreciate the responses! Steve, I agree that journalists are suspect. But we in the tech journalism/tech writing trades like to think that we're a cut above.

Reid and William, I'll take a look at those other forums you mention.

And thanks, Jeanne, for that link to one of Esther's threads. She really did speak highly of JavaRanch as one of the true developer-focused online communities -- "community in the best sense of the word," she said.

One caveat that I'm not a developer so my questions may occasionally (or often) be dumb or simplistic. As for post-deployment performance tuning, the few folks I've asked so far have told me that the topic itself is a bit odd. Tune and test early and often is the mantra.

I agree, but inevitably there are situations where folks want to improve the performance of code already in production. (Plus, that's the assignment from my editor.) I've also heard that in most post-deployment situations, you're really stuck with tuning the hardware and not the code. Is that accurate? Surely there must be some cases where developers are called in to help squeeze more performance and efficiency out of a given application? Or, how can you build features into your code to make it easier for the users themselves (or at least their local IT managers) to tune the application?

Like Esther, I'm looking mostly for anecdotes and stories from hands-on developers working in code day in and day out. Being a journalist (sorry, Steve) I'm looking for folks willing to go on the record (ie, willing to be named in the column, and also to name company or product names). But I realize that this isn't always possible (especially if you're talking about how dramatically things went wrong on a particular project, etc.), so I'm more than happy to negotiate attributions to something you're comfortable with. For example, "...according to a Chicago-based J2EE specialist and 10-year programmer..."

Thanks again!
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
OK, so I figured out how to delete my new thread. I will get more savvy... maybe. Thanks.
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Gang, I'm getting ready to write so I thought I try back. I probably haven't done a good job posing a thought-provoking question. Here's a stab at one (and a half): Is post deployment performance testing/tuning ever a good idea? If so, when? Thanks!
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 852
Is post deployment performance testing/tuning ever a good idea?

Yes, I think this is very important. But, I would like to take it one step further.

I believe in CONTINUAL AUTOMATED monitoring in production - not simply pointing a test script at production when you first deploy and ending it there. Most application developers have no real idea what the true load is, what peak times are and how users interact with the system. There is no better load/performance testing than your real users in action.

Where I work we collect real scalability and performance metrics via JAMon (http://www.jamonapi.com) in production. Some of the metrics collected are max simultaneous users, and max simultaneous http requests as well as performance stats under load. All of this information is collected automatically as users interact with the system. This information can be further used in test to derive realistic load tests.

Another thing that becomes possible with real-time monitoring is alerts (emails/pages) based on events (error/failed login etc) occuring within your application. For example if a log4j error log method is called or an exception is thrown send an email to the application admin staff (this beats combing logs).

Other possible alerts are performance degradation such as page response time exceeding 5 seconds, or some resource threshold being exceeded (db or http sessions getting close to the servers configured limits). Other types of alerts include:
- failed logins (a possible security breach)
- SQL errors
- Purchases exceeding a threshold for the day (say $1 million)
- File handles getting close to configured limits, cpu running at 100%, or memory running low

Of course there are many more possibilities.

I am expanding the scope of your question - but, I view the traditional view of "performance monitoring" to be a special case of the more general (and useful) case of "application monitoring". I also view the form of monitoring where testing staff hammer on the system to be less valuable than the concept of an application proactively monitoring all aspects of itself and notifying admin staff when events of interest occur.

JAMon statistics are kept in memory and using another of my API's (ArraySQL) you can query the contents of JAMon and take appropriate action if data matches criterian (For example Send an email if a row is returned by: select * from array where label='open db connections' and total>50)

Read the Performance FAQ below for more thoughts on monitoring in production. Also read the "what's new in jamon 2.0?" discussion for more thoughts on the general concept of "application monintoring" via JAMon.

My hope is that one day all applications will self-monitor - for performance, errors, and any other events of interest that may occur.
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Thanks, Steve. I appreciate the reply. First things first. May I quote you? If so, how can I refer to you (ie, can I name your employer, geographic location, # of years as a developer, etc.)

I think that's a good point and a good way to expand the original question. I guess my follow-up: what do you do with all the data you gather as you continually monitor your application? My sense is that most of the quick fixes for problems that might be identified involve hardware (adding CPUs, memory, etc.) Is that accurate? Are there any easy, obvious ways to iteratively tweak and improve code, or do most of the problems identified in a continuous monitoring situation have to wait until the next release cycle to be fixed? Perhaps as we move more to software-as-a-service norms, continual monitoring will gain traction (as firms more easily will be able to iteratively upgrade their code).

Thanks again.
Jeanne Boyarsky
internet detective
Marshal

Joined: May 26, 2003
Posts: 29220
    
135

Originally posted by Geoff Koch:
Is post deployment performance testing/tuning ever a good idea? If so, when?

Definitely a thought provoking question!

For the purposes of this thread, I assume you mean deployment to production rather than development.

I think post deployment performance testing/tuning is a good idea so long as it isn't used instead of pre-production testing/tuning. "We ran out of time" doesn't do much for real tuning. When done in addition, you can see how things function in the actual production scenarios and over time. For example, a test after production can show resource issues due to a lot of new applications on the server.

- Jeanne - A Java Developer in New York City
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 852
Yes, you may quote me. Refer to me as Steve Souza, a Sybase Principal Consultant based out of Washington DC with 20 years of software development experience.


<<what do you do with all the data you gather as you continually monitor your application? >>
Right now we collect aggregate stats (i.e. throw out detailed data points but keep summaries). We do not save them to a database, but that would be best long term.

<<My sense is that most of the quick fixes for problems that might be identified involve hardware (adding CPUs, memory, etc.) Is that accurate? >>
Probably true, but the most improvement is usually attained by logic changes. For most apps I work with the database is the bottleneck. Sometimes I have seen adding an index or 2 can improve performance much much more than increasing CPU's or memory. Often indexes can be added without redeploying the application. See the faq below for more info on post production monitoring.

<<Perhaps as we move more to software-as-a-service norms, continual monitoring will gain traction (as firms more easily will be able to iteratively upgrade their code).>>
Servlets, ejb's and stored procedures are often also easy to redeploy.
[ April 09, 2006: Message edited by: steve souza ]
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Thanks a lot, Steve. I appreciate the info!

You, too, Jeanne. And yes -- I'm trying to focus on deployment to production.
kri shan
Ranch Hand

Joined: Apr 08, 2004
Posts: 1367
Hi Steve Souza,

Where I work we collect real scalability and performance metrics via JAMon (http://www.jamonapi.com) in production. Some of the metrics collected are max simultaneous users, and max simultaneous http requests as well as performance stats under load. All of this information is collected automatically as users interact with the system. This information can be further used in test to derive realistic load tests.

In this paragraph what is the diference between max simultaneous users and max simultaneous http requests ?
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 852
In this paragraph what is the diference between max simultaneous users and max simultaneous http requests ?

By 'max simultaneous users' I meant the maximum ammount of simultaneous http sessions. You can also tell the average, and current number of simultaneous http sessions.

By 'max simultaneous http requests' I meant the maximum ammount of http page requests that occurred concurrently/simultaneously. You can also tell the average and current number of simultaneous http page requests.

All pretty handy for figuring out concurrency.
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 852
Here is a link to the article Geoff wrote. It is on page 36

http://www.stpmag.com/issues/stp-2006-06.pdf
Geoff Koch
Ranch Hand

Joined: Mar 19, 2006
Posts: 35
Thanks, Steve, for doing posting that!
steve souza
Ranch Hand

Joined: Jun 26, 2002
Posts: 852
No problem. What is your next article?
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: tech journalist seeking comments of post-deployment performance tuning
 
Similar Threads
STP Mag columnist seeking comments on post-deployment tuning
References for Java Performance Optimization
This week's giveaway!
seeking tips on posts-deployment tuning for st&p magazine
difference between deployment and installation