aspose file tools*
The moose likes Agile and Other Processes and the fly likes Cost of Change Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of Spring in Action this week in the Spring forum!
JavaRanch » Java Forums » Engineering » Agile and Other Processes
Bookmark "Cost of Change" Watch "Cost of Change" New topic
Author

Cost of Change

Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
Hi, please see if what I wrote on the cost of change makes any sense. It talks a lot about reducing costs, but I think I'm really trying to convince some people not to increase the cost with more and more method stuff. Consider it a rough draft, completed right when I should be asleep.
[ June 14, 2007: Message edited by: Stan James ]

A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
I haven't yet found the time to really read what you wrote. So I'm not sure whether the following is helpful at all...

What I've found when talking about this topic is that there are actually two different cost of change curves, and that they get often confused for one another:

- the cost of fixing a bug dependend on when in the lifecycle it is found, and

- the cost of introducing a new feature, depending on when in the life of a project it is identified.

It seems to me that the former curve still is exponential, although probably with a significantly small exponent.

The latter curve, I feel doesn't have to be. Using Agile techniques or similar, it rather feels like the cost of adding a new feature is more or less constant over the lifetime of a project.

Does that make sense?


The soul is dyed the color of its thoughts. Think only on those things that are in line with your principles and can bear the light of day. The content of your character is your choice. Day by day, what you do is who you become. Your integrity is your destiny - it is the light that guides your way. - Heraclitus
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
Yup. Rereading, I think I don't think the page makes clear where I'm trying to go with this. Our Project Management Office flashes up this slide and then talks about all the process they want to add to firm up requirements. Of course agilists believe this is not going to work, but I think it ought to show up as not cost effective even if the fantasy of eliminating change were true. It increases the cost and time to market of everything too much.
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Yes. I think the Lean vocabulary is very well suited to appeal to management in that regard. Eliminating waste by reducing inventory, optimizing the value stream and such...
Frank Carver
Sheriff

Joined: Jan 07, 1999
Posts: 6920
One of the approaches I take when dealing with this issue is to use the power of the traditional cost-of-change curve itself. The logic goes something like this:


Let's take as given that the later we discover a bug, the more expensive it is to fix. So we obviously need to discover any bugs as early as possible.

How do we discover a bug? Bugs only show up when we test a solution. If there's no code to test, or no test to perform, we can't tell if there are any bugs or not. So, to catch bugs as early as possible, we need code and tests in place as early as possible.

But, if we wait for the whole solution to be coded and all the test cases to be written, we are waiting until almost the whole way through the project. We are not getting any of the benefits of catching bugs early.

The way round this conundrum is to aplit the solution into small bits. If we code and test one small bit right at the start of the project, any bugs in that bit will be found and fixed _really_ early, when they are extremely cheap to fix. Then we can code and test another small bit. Any bugs in that bit will be almost as cheap to fix as the bugs we found in the first bit.

When we have coded and tested the first bit, and we integrate the two small bits together, we'll know that we can trust the first bit, because its bugs have already been found. As we code, test, and integrate each new small bit, we get closer and closer to the final solution. At each stage, as many bugs as possible are found as early as possible.

Even when we are getting toward the end of the solution, and bugs in new small bits are proportionally more expensive to fix, the overall cost is still much lower, because such a large proportion of the bugs have already been found and fixed while they were almost free.


Does that sound reasonable?


Read about me at frankcarver.me ~ Raspberry Alpha Omega ~ Frank's Punchbarrel Blog
Gabriel Claramunt
Ranch Hand

Joined: May 26, 2007
Posts: 375

My only comment is almost all the modern development process are iterative, not only the agile ones, and I expect similar results from any iterative process agile or not...


Gabriel
Software Surgeon
Frank Carver
Sheriff

Joined: Jan 07, 1999
Posts: 6920
Originally posted by Gabriel Claramunt:
My only comment is almost all the modern development process are iterative, not only the agile ones, and I expect similar results from any iterative process agile or not...


I agree in principle. Unfortunately, I've also seen many implementations of such development processes where the iteration is only in the development phase. All the requirements are still gathered up-front into some huge requirements document before any development begins, pushing all the development into the more expensive part of the cost curve.

I'm currently studying a unit on requirements for a MSc course, and on the very first page of the set text book(1) it states:

"Discovering the requirements during construction, or worse, when your client starts using the product, is so expensive and so inefficient, that we will assume that no right-thinking person would do it and will not mention it again"


This is the kind of thinking which is being taught in universities right now.

(1) Mastering the Requirements Process, Robertson and Robertson, Addison Wesley 1999, or more details on the second edition, which I've not seen
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
almost all the modern development process are iterative


Sigh, if only we were allowed to use one. As Frank says, many people say "iterative" but don't do anything approaching what the agilists think of. I borrowed this definition somewhere:

Too often it means "We had to cut some features to make the date and we'll get to them later."

Back to topic ... Agile doesn't typically sell itself on cost savings, does it? Reducing waste is in there, building the most valuable thing first so you can stop if you want ... those ideas apply to cost.
Gabriel Claramunt
Ranch Hand

Joined: May 26, 2007
Posts: 375

I agree in principle. Unfortunately, I've also seen many implementations of such development processes where the iteration is only in the development phase. All the requirements are still gathered up-front into some huge requirements document before any development begins, pushing all the development into the more expensive part of the cost curve.


I've seen even a manager that was doing iterative requirements, then iterative design, and then iterative coding... and was calling that RUP, go figure!

No process is immune to bad implementations: guess it would be easy to find many "Agile" implementations that are just code & fix... specially risky is XP because of the high discipline required to following the practices.

For the requirements course, one shouldn't had to be at MSc level to learn about it... it should be taught at BsC level, every software developer must know how to handle them
And the quote is not that bad , consider a XP project: the requirements discovery is delegated to the onsite customer, what happens when you have the team ready to pair-program the tests required to start coding, but you don't have user story cards because the customer is trying to discover what it needs to be constructed? For me is expensive having a full team just waiting. And agile or not, don't you think is expensive to develop a beautiful product that nobody needs?
(BTW, is not far from what happened to the original XP project, the CCC)
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Gabriel Claramunt:
My only comment is almost all the modern development process are iterative, not only the agile ones, and I expect similar results from any iterative process agile or not...


Where Agile approaches are special, as far as I can tell, are the preferred size of the iterations (= very small), the focus on producing a production ready system from iteration one, and the focus on collaboration.
Frank Carver
Sheriff

Joined: Jan 07, 1999
Posts: 6920
Originally posted by Gabriel Claramunt:
And the quote is not that bad , consider a XP project: the requirements discovery is delegated to the onsite customer, what happens when you have the team ready to pair-program the tests required to start coding, but you don't have user story cards because the customer is trying to discover what it needs to be constructed? For me is expensive having a full team just waiting.


I think you may be missing the point of an agile approach. The idea is that you don't need to know all the requirements before starting design/code/test. As soon as the development team has even a single requirement, however small, development work can begin. As soon as it's completed it can feed back into the requirements process, In the meanwhile the customer/requirements folks will have had time to identify and agree more needs.

In my experience there is a lot less waiting around in an agile development process.

And agile or not, don't you think is expensive to develop a beautiful product that nobody needs?

Sure. But again, in my experience an iterative, agile approach provides early feedback so customers can see exactly what they are getting and responds quickly to changing needs. This helps counteract the tendencies toward "gold-plating" and "design by committee" faced by up-front requirements process.

Have you experienced agile practitioners falling into these sorts of traps?
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Gabriel Claramunt:

consider a XP project: the requirements discovery is delegated to the onsite customer, what happens when you have the team ready to pair-program the tests required to start coding, but you don't have user story cards because the customer is trying to discover what it needs to be constructed? For me is expensive having a full team just waiting. And agile or not, don't you think is expensive to develop a beautiful product that nobody needs?
(BTW, is not far from what happened to the original XP project, the CCC)


As far as I can tell, the reasons for the C3 project being cancelled were rather complex. And the C3 team certainly did not work without the customer providing stories, and the system actually was in production for a couple of years.
Gabriel Claramunt
Ranch Hand

Joined: May 26, 2007
Posts: 375

Originally posted by Ilja Preuss:


As far as I can tell, the reasons for the C3 project being cancelled were rather complex. And the C3 team certainly did not work without the customer providing stories, and the system actually was in production for a couple of years.



I know the reasons are complex, I didn't mean that in C3 the team worked without stories, the comment was more about the second part: maybe it was developed with an excellent process, but AFAIK it didn't replaced the system it was intended to replace... that's why I said "a beautiful product that nobody uses" is not far from what happened to C3
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Gabriel Claramunt:

I know the reasons are complex, I didn't mean that in C3 the team worked without stories, the comment was more about the second part: maybe it was developed with an excellent process, but AFAIK it didn't replaced the system it was intended to replace... that's why I said "a beautiful product that nobody uses" is not far from what happened to C3


As far as I know, the system was in production for something like two years, and at its peak paid 80% of Chrysler's employees. That seems to be "far from" "nobody uses" to me - your mileage may vary.

The two top reasons for C3 being canceled that I hear from those involved are

- a misalignment between the goals of the Goal Doner and the Gold Owner, and
- a significant change in the personal of top management, with the new management deciding that the company should focus on its core competencies
Gabriel Claramunt
Ranch Hand

Joined: May 26, 2007
Posts: 375

Some process, with higher focus on requirements have an explicit goal of align all stakeholders to avoid (or minimize) situations like "a misalignment between the goals of the Goal Doner and the Gold Owner", because they're costly for the organization and fatal for the project.
AFAIK requirements is not just writing down what the user wants (that is a good source of big failures).
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
Some process, with higher focus on requirements ...


Yup, there better be business processes going on above XP to take care of this kind of problem. XP doesn't claim to run the company, just to produce the software it's immediate customers want.

It's way too common for big companies (and maybe small ones, but I haven't been there to see) to launch major software initiatives for seriously wrong reasons. Years ago I saw several of these grind on to completion even though everybody knew nobody would ever use them. (I think I only built one, myself.) Everybody proudly declared success on completion and quietly swept the debris under the rug. More recently I see a lot of emphasis on "portfolio management" and the new buzzword "governance" which address this kind of mess directly. I'm sure my company is quite a bit smarter than it was twenty years ago.

So for me the end state of Chrysler's C3 is no surprise at all, and so typical of ginormous companies that I don't figure it reflects on XP at all.
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
BTW: I'm going to pull the page I referenced at the start of all this. After Googling a bit I realized Ron Jeffries made the same point much better in 2000.
Gabriel Claramunt
Ranch Hand

Joined: May 26, 2007
Posts: 375

Just a little correction, the article is from: Alistair Cockburn
Always a very interesting read.
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Gabriel Claramunt:
Some process, with higher focus on requirements have an explicit goal of align all stakeholders to avoid (or minimize) situations like "a misalignment between the goals of the Goal Doner and the Gold Owner", because they're costly for the organization and fatal for the project.
AFAIK requirements is not just writing down what the user wants (that is a good source of big failures).


XP has that goal, too. And it works to achieve it by close collaboration with all stakeholders, and early feedback by frequent delivery of the actual product. If you know of a better way to achieve it, I'd seriously like to hear about it.

In the C3 project, there actually were situations where the team would tell the project manager that some goal couldn't be reached, and the project manager answered "I can't tell that to my boss". That's a people problem, and I'm not aware of any process that can fix it. It's the people on the project that have to fix it. In that specific case, it seems like they didn't.
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
XP has that goal, too. And it works to achieve it by close collaboration with all stakeholders, and early feedback by frequent delivery of the actual product. If you know of a better way to achieve it, I'd seriously like to hear about it.


Defining "all stakeholders" is key here. An XP team might not even be aware that they're being isolated from important folks. I've seen many cases of systems being built in different parts of the company with huge overlap in functionality and the business problem they think they're solving. Some stakeholder with the ability to see the waste and kill one of the projects was missing or out to lunch or maybe just watching the teams compete for political points. Humans have many motivations besides doing the best thing for the company. I'd say management of that kind of thing is outside XP scope.
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Stan James:
I'd say management of that kind of thing is outside XP scope.


I agree in the case of "XP as a methodology" - that is, there are no practices in XP that tackle this problem.

On the other hand, the values and principles probably apply there, too, and it's certainly something that the XP community is discussing.

BTW, regarding the cancellation of C3, there recently was an interesting post at the xp mailing list: http://tech.groups.yahoo.com/group/extremeprogramming/message/131252
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
I'm in a group now that has a lot of churn in plans right up to delivery. It's at a higher level than shuffling XP story cards and it's a feature of the business, not a failure on anybody's part. But they're trying to "fix" it by nailing down requirements better up front, which seems like exactly the wrong answer these days. I keep wondering if the Scrum backlog and iteration ideas would scale up to this larger scope, or if they should maybe abandon the larger scope and prioritize at a finer grain.

I kept some of that discussion about C3. It's interesting what differing accounts you can find for the same events. Some of the detailed comments about who played what kind of political roles before and after C3 go right to this higher level planning and support for larger grained projects. It isn't easy for sure.
 
I agree. Here's the link: http://aspose.com/file-tools
 
subject: Cost of Change