Q to Jack: Continuous Performance

Jack,
some time ago on theServerSide.com was posted an article "Continuous Performance: The Next Advance in Software Development"...its' main idea was that testing performance several times a day(as part of build process maybe) makes performance tuning of the final product vastly more effecient.
What do you think of such a tactics in the implementation cycle?
Regards, Stepan.

Well, I am not Jack - but I would generally think that the earlier and more often you can manage to get feedback about the fulfillment of requirement, the better.
Is there something to be said *against* it?

Looks like someone took what I've always advocated and applied it XP-style :-p
I suspect, while good in principle, it's not yet ready for prime time, in that we don't yet have good ways to automate performance tests in most cases (web applications being an exception). I also think people new to it might be confused or would misuse the data--not that that is a reason not to do it, just a reason to proceed with caution.
--Mark

Originally posted by Ilja Preuss:
Is there something to be said *against* it?

The main problem I think is to understand the need for such practice. It's not easy to explain to the management why you need to spend time on developing scripts for doing continuous performance. It depends on the size and length of the project very much. And I don't know any resource where I can see statistics of this practice' usage, when it's appropriate, when it's not.

It's an issue of resources. If you are performance testing unstable work, you will start tuning parts of the code that will never otherwise need tuning.
And when do you stop? With functional testing, you stop fixing when the test doesn't produce an error. How do you know when to stop tuning? For the whole application, you should have performance targets, but what are going to do, give yourself a target for every piece of functional code you produce? MethodX does X and takes 124 nanoseconds. MethodY does Y and takes 15 milliseconds. That's fun.
Look for design bottlenecks. Look for architecture bottlenecks. Specify performance targets and profile stable units looking for implementation bottlenecks using those specifications as benchmarks.
Stability is very different from performance. If a path followed once or twice by the application fails, the application is breaks. But is a path followed once or twice by the application is slow, mostly you don't even notice.
--Jack Shirazi
JavaPerformanceTuning.com

Zen and the Art of Performance Tuning..
Every water tower can be filled one spoonfull at a time.

...unless it's leaking. Which may be the case on some projects.

And we wouldn't want to be using spoonfuls if there's a nice bucket lying nearby...