aspose file tools*
The moose likes Agile and Other Processes and the fly likes Agile Fixed cost development? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login
JavaRanch » Java Forums » Engineering » Agile and Other Processes
Bookmark "Agile Fixed cost development?" Watch "Agile Fixed cost development?" New topic
Author

Agile Fixed cost development?

Chris Nappin
Ranch Hand

Joined: Aug 04, 2005
Posts: 36
Here is an open question I hope will trigger some interesting discussion:

How can Agile methodologies be best put to use when doing fixed cost development projects? javascript: x()
Confused

Most market sector I've worked in has involved doing project work for a fixed up-front cost, fixed up-front time frame and a mutually agreed project scope. Working on a less well defined basis, or even working on a T&M basis, was not possible. Rarely are all of those up-front targets ever met, by the way.

The best way (i.e. meets customer's expectations and makes money for my employers) to do such projects seems to be to use "old fashioned" methodologies and a detailed up-front design and estimation. Going agile tends to be counter-intuitive and results in much more risk and generally making no money for my employers!

I guess a best approach might be a mixture of agile and more traditional methodologies, however that's not a message I've ever heard any pragmatic Agile evangalist ever preach?


Technical Architect, SCJP, SCWCD
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
I had a very long answer that I'm editing to replace with questions. Compared to a traditional team ...

Why would an agile team deliver a less accurate estimate?

Why would an agile team deliver less software in the same fixed time?

Do these get to the heart of your doubts?
[ July 04, 2006: Message edited by: Stan James ]

A good question is never answered. It is not a bolt to be tightened into place but a seed to be planted and to bear more seed toward the hope of greening the landscape of the idea. John Ciardi
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
To add to what Stan already said...


Most market sector I've worked in has involved doing project work for a fixed up-front cost, fixed up-front time frame and a mutually agreed project scope.


So far, that's not at all counter to Agile practices. Fixed time and cost is not a problem at all. A roughly agreed on scope can be done as usual, too.

What Agile adds to the mix is the ability to give you earlier return on investment by early and frequent releases, to give you a better sense of progress (and thereby actually the ability to *manage* the project) and to allow you to adjust scope in the small.

The last point is an important one: even if scope is fixed, there are most often different ways a feature can be implemented, and important and less important parts of features. With other words, even if scope is fixed in the large, you can still do a lot of good by managing scope in the small.

The best way (i.e. meets customer's expectations and makes money for my employers) to do such projects seems to be to use "old fashioned" methodologies and a detailed up-front design and estimation. Going agile tends to be counter-intuitive and results in much more risk and generally making no money for my employers!


Why do you think that an Agile approach would result in *more* risk? And in risk for whom?

I guess a best approach might be a mixture of agile and more traditional methodologies, however that's not a message I've ever heard any pragmatic Agile evangalist ever preach?


What would such a mixture look like, and what risks would it address how?


The soul is dyed the color of its thoughts. Think only on those things that are in line with your principles and can bear the light of day. The content of your character is your choice. Day by day, what you do is who you become. Your integrity is your destiny - it is the light that guides your way. - Heraclitus
Rajah Nagur
Ranch Hand

Joined: Nov 06, 2002
Posts: 239
As far as estimation is concerned, the beauty of Agile process is that the developer estimates the amount of time that will be taken to develop his module.

But in fixed bid projects, since the estimation is decided upfront, stirictly sticking to Agile methodology becomes a bottleneck.

The best bet would be to use the hybrid model i.e. take the best practices of Agile process (pair programming, TDD, continous integration, lightweight QA process ) and combine with say RUP (process workflow, activities definiton, estimation and template guidelines etc).

No single methodology fits any project. Tailoring needs to be done.

The end result should be always focussed on the successfull timely delivery of the project..(if all goes well and project is delivered then it really does not matter/nobody cares what process/ methodology was used).


You can't wake a person who is <b><i>pretending</i></b> to be asleep.<br />Like what <b>"it"</b> does not like - <i> Gurdjieff </i>
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Rajah Nagur:
[QB]As far as estimation is concerned, the beauty of Agile process is that the developer estimates the amount of time that will be taken to develop his module.

But in fixed bid projects, since the estimation is decided upfront, stirictly sticking to Agile methodology becomes a bottleneck.


I'm not following you. In which way would "sticking to Agile" be a bottleneck? And how do you get a good estimatio upfront in a fixed bid project other than by asking the developers?

The best bet would be to use the hybrid model i.e. take the best practices of Agile process (pair programming, TDD, continous integration, lightweight QA process ) and combine with say RUP (process workflow, activities definiton, estimation and template guidelines etc).


Well, from the standpoint of practices, many Agile processes (for example XP) actually *are* instances of RUP.

On the other hand, what really is important about Agile are the values, the philosophy. And to me those are in conflict with what seem to be the values of RUP.

So in neither way saying to "combine Agile and RUP" makes much sense to me.

No single methodology fits any project. Tailoring needs to be done.


Very true. But before you can tailor effectively, you need to know very well the original thing. And the best way to know it is to have it tried in its pure form for a while.

The end result should be always focussed on the successfull timely delivery of the project..(if all goes well and project is delivered then it really does not matter/nobody cares what process/ methodology was used).


Again true. That is, until your competitor beats you by being even better using a radically different approach.
Chris Nappin
Ranch Hand

Joined: Aug 04, 2005
Posts: 36
Thanks for the various replies.

To explain my issue a bit further, to make money on a fixed cost contract the estimate must be very accurate. Any overspend eats up the profit margin until the project is being done for considerable cost to the software development organisation.

The cost of a project is agreed well before any developers get involved - usually only sales, pre-sales consultants and architects would be involved. This would be the "Inception" phase in RUP, right? These people have to produce an accurate estimate based on the scope of the project.

My question about agile is this - if deliberatly choosing a methodology in which the scope of the project is not known, and capturing detailed requirements and doing software design etc. is left until later in the project, how can the estimates be anything but more inacurate than the more "old fashioned" approaches?

No doubt I'm completely missing the point, but capturing the entire requirements in minute detail, then doing a detailed design "in stone" (and charging for "Change Requests" if the customer changes their mind) would be far more likely to turn a profit for the software development organisation.

Let me introduce a couple of solutions that I've heard of (but not seen in practice), does anyone know of any others?

1. The fixed cost buys a fixed amount of time, which is estimated to be sufficient for the customer's major functionality items but can't be guaranteed. Further discussion starts when the alloted amount of time runs out. Not something I know customers will accept, functionality is usually part of the contract.

2. Each major iteration is a separate contract with an individual price. Here agile comes to it's strengths as each successive iteration can be priced more accurately than the last, as the overall projects proceeds. Again, not something many customers will accept, as they have no idea what cost they are getting into when starting a project.
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Chris Nappin:
To explain my issue a bit further, to make money on a fixed cost contract the estimate must be very accurate.


Are you saying that non-Agile approaches are well-known for accurate estimates?

If it's working that well for you, why are you interested in Agile development?

Any overspend eats up the profit margin until the project is being done for considerable cost to the software development organisation.


Yes. Which forces the development organization to interprete the contract to the disadvantage of the customer, and to compromise quality. It's a lose-lose situation.


The cost of a project is agreed well before any developers get involved - usually only sales, pre-sales consultants and architects would be involved.


How well is that working for you?

This would be the "Inception" phase in RUP, right? These people have to produce an accurate estimate based on the scope of the project.


I'm not a RUP expert, but I've heard that the output of every phase, including inception, includes running software - some kind of prototype or spike or similar for this phase, I'd guess.

My question about agile is this - if deliberatly choosing a methodology in which the scope of the project is not known


This is a common misunderstanding. At the start of an Agile project, the scope of the project is as well known as at the beginning of any other project. It's just that you don't drill that much into all the details before you start coding, because we expect the details to change, anyway.

and capturing detailed requirements and doing software design etc. is left until later in the project, how can the estimates be anything but more inacurate than the more "old fashioned" approaches?


In the time an "old fashioned" team has captured the detailed requirements and done the design, an Agile team has captured the rough requirements and already implemented the most important ones, verified by the customer.

Why should the latter be more inacurate?

No doubt I'm completely missing the point, but capturing the entire requirements in minute detail, then doing a detailed design "in stone" (and charging for "Change Requests" if the customer changes their mind) would be far more likely to turn a profit for the software development organisation.


Why should providing less value to the customer lead to more profit?

Let me introduce a couple of solutions that I've heard of (but not seen in practice), does anyone know of any others?

1. The fixed cost buys a fixed amount of time, which is estimated to be sufficient for the customer's major functionality items but can't be guaranteed. Further discussion starts when the alloted amount of time runs out. Not something I know customers will accept, functionality is usually part of the contract.


That's exactly what we did in a recent one-year project.

Each major iteration is a separate contract with an individual price. Here agile comes to it's strengths as each successive iteration can be priced more accurately than the last, as the overall projects proceeds. Again, not something many customers will accept, as they have no idea what cost they are getting into when starting a project.


In fact they still have full control over cost. What they don't know exactly is what they will get for their money. But they are used to that - even with fixed price contracts, most often they don't get what they actually want. The nice thing about Agile development is that they see much earlier what they will get, and are in a much better position to steer the project. Most customers understand that, in my experience.
Scott Ambler
author
Ranch Hand

Joined: Dec 12, 2003
Posts: 608
The cost of a project is agreed well before any developers get involved - usually only sales, pre-sales consultants and architects would be involved. This would be the "Inception" phase in RUP, right? These people have to produce an accurate estimate based on the scope of the project.


No, this is not the Inception phase in RUP. During the Inception phase you do some requirements modeling, perhaps to the "20% level" and an initial estimate, amongst other things.


My question about agile is this - if deliberatly choosing a methodology in which the scope of the project is not known, and capturing detailed requirements and doing software design etc. is left until later in the project, how can the estimates be anything but more inacurate than the more "old fashioned" approaches?


The older approaches have been shown to be very poor in practice. Estimating up front has been shown to be far less accurate than estimating on a just in time basis, which is why estimators suggest that you do a large range at the beginning of the project and then narrow that range as you gain more knowledge throughout the project. If you were to do a bit of research into estimating you would quickly discover these things.


No doubt I'm completely missing the point, but capturing the entire requirements in minute detail, then doing a detailed design "in stone" (and charging for "Change Requests" if the customer changes their mind) would be far more likely to turn a profit for the software development organisation.


Actually, that's been shown to be a very risky way to work. It's bad for the IT folks and it's bad for the customers. Doing the requirements up front, something called BRUF, has been shown to result in a 45% wastage on average. Does throwing away nearly half your budget on useless functionality sound like a good approach to you?

You might find Comparing Approaches to Budgeting and Estimating Software Development Projects to be of interest.

- Scott


<a href="http://www-306.ibm.com/software/rational/bios/ambler.html" target="_blank" rel="nofollow">Scott W. Ambler</a><br />Practice Leader Agile Development, IBM Rational<br /> <br />Now available: <a href="http://www.ambysoft.com/books/refactoringDatabases.html" target="_blank" rel="nofollow">Refactoring Databases: Evolutionary Database Design</a>
Chris Nappin
Ranch Hand

Joined: Aug 04, 2005
Posts: 36
A few interesting responses. I get the impression that the initial estimation process (part of getting the sale) is generally not any different regardless of the methodology being used. It's often this step that defeats projects before they start, so I'd be interested in any thoughts on improving this step.

And agile project management only deal with issues "in the small". Any estimates made during later phases of an agile project might be more accurate but have to fit into the original overall cost or timeframe, which could be wildly wrong.

I'm intersted in agile methodologies because I've witnessed a fair few projects fail on which I think agile approaches would have helped. However I'm still not sure how an agile project can be sold, on a fixed cost, fixed duration, fixed scope basis.

It's easy to explain to a customer who's been badly burnt by failing projects that agile approaches make sense and fixed cost/duration/scope is unrealistic, however for everyone else the lowest cost seems to win every time.

Originally posted by Scott Ambler:


You might find Comparing Approaches to Budgeting and Estimating Software Development Projects to be of interest.

- Scott


Thanks Scott, I'll have a look. I notice this includes a "compromise" middle ground between agile and traditional approaches, which was my original question.
Stan James
(instanceof Sidekick)
Ranch Hand

Joined: Jan 29, 2003
Posts: 8791
I get the impression that the initial estimation process (part of getting the sale) is generally not any different regardless of the methodology being used.


That's what I was probing for at the beginning! At work I laid out the waterfall steps defined for our organization on a timeline and put an agile alternative below it. The inception and estimation phases didn't move. The actual techniques differ in that agile folks estimate by finer grained stories which we have found to give much better estimates. But if you have officer level folks and an architect doing this bit, it won't be better or worse with agile techniques. It will be really bad, period.

The two timelines converge at the end, too, around deployment. It's rather a big deal here, not just a regular Friday build or something.

All the stuff in the middle of the two timelines is different. One has requirements gathering and design phases, solidly closed out with official document signoffs, then construction and QA phases. The other has some number of iterations. Here's where we have to compare agile and traditional.

Again, why would agile teams deliver less software in this time? And this time it's a trick question ... I can't imagine any reason.

Even if we don't assert that an agile team will deliver more software in this time, they will work in smaller tasks that are easier to predict and measure and they will deliver some amount of true business value in working software instead of a pile of documents. When it comes to the crunch and we have to sacrifice the date or some scope the agile team will have delivered the higest value first and the lowest value scope falls off the table naturally. The hard choices have already been made collaboratively. In fact, the visibility into progress is so good that everybody sees what's coming far in advance and nobody calls it a "crunch" decision.
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Chris Nappin:
A few interesting responses. I get the impression that the initial estimation process (part of getting the sale) is generally not any different regardless of the methodology being used. It's often this step that defeats projects before they start, so I'd be interested in any thoughts on improving this step.


Well, from the discussion until today, I'd wager that you could get some good improvement by having the first estimate made by the same people who later will have to implement it.

And agile project management only deal with issues "in the small". Any estimates made during later phases of an agile project might be more accurate but have to fit into the original overall cost or timeframe, which could be wildly wrong.


The same is, of course, true for non-Agile projects. That's why estimates are called estimates, not facts.

The advantage of Agile projects is that you get more accurate feedback earlier, which allows you to actually *manage* the project - by managing the available resources, by managing the details of the scope, by managing how the work is done.

I'm intersted in agile methodologies because I've witnessed a fair few projects fail on which I think agile approaches would have helped. However I'm still not sure how an agile project can be sold, on a fixed cost, fixed duration, fixed scope basis.


I'm still not sure what you think would happen if you sold the project the usual way, and then implemented it in an Agile way.

It's easy to explain to a customer who's been badly burnt by failing projects that agile approaches make sense and fixed cost/duration/scope is unrealistic, however for everyone else the lowest cost seems to win every time.


Tell you what, Agile approaches are a really good way to provide more value at lower costs.

One problem with fixed price contracts is that, because changes are so costly to the customer, he is forced to put all kind of features into the contract, just in case. That leads to the feature bloat Stan described above - a lot of the features developed don't really provide value, but are just there because of the fear of changes.

Hope this helps...
Rajah Nagur
Ranch Hand

Joined: Nov 06, 2002
Posts: 239
Couple of things are not clear:

It is very ideal situation if the programmers/developers contribute to the estimation of the project.

But -
In many *practical* situations, when the project is in proposal stage/funding stage/RFP stage, there is no team existing. A rudimentary estimation has to happen for the scope/size/tenure of the project. After this the programmers/PLs/designers/analysts join this project. This estimation happens based on person's experience.

Second point:
Since most of the projects in an accout will be the similar nature (For e.g. healthcare, financial services, online ecommerce accounts etc) and based on the prior experience many can come up with fairly accurate estimate behorehand and then manage the team and finish the deliverables.

This is what is happening in most of the cases..a highly experienced team (interacting with the customer) estimate the project timelines and then recruit/choose the team and give the deadlines. The team can then choose any methodology to execute project but within the limits of the timelines set.

...just my thoughts..
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Rajah Nagur:
It is very ideal situation if the programmers/developers contribute to the estimation of the project.


If it's ideal, it's probably something we should strive for, isn't it?


In many *practical* situations, when the project is in proposal stage/funding stage/RFP stage, there is no team existing. A rudimentary estimation has to happen for the scope/size/tenure of the project. After this the programmers/PLs/designers/analysts join this project.


If you do have a rough idea of who might later join the project, you could try to get at least some of the future team members to help in the estimation. Might be worth a try, I'd guess.

Since most of the projects in an accout will be the similar nature (For e.g. healthcare, financial services, online ecommerce accounts etc) and based on the prior experience many can come up with fairly accurate estimate behorehand and then manage the team and finish the deliverables.


Well, if that is working well for you, there is no reason to change I guess.

Is it working well for you?

This is what is happening in most of the cases..a highly experienced team (interacting with the customer) estimate the project timelines and then recruit/choose the team and give the deadlines. The team can then choose any methodology to execute project but within the limits of the timelines set.


This seems to imply that the team you choose doesn't have an impact on how long the project will take. That's not exactly my experience, but again, if it's working for you...
Scott Ambler
author
Ranch Hand

Joined: Dec 12, 2003
Posts: 608
In many *practical* situations, when the project is in proposal stage/funding stage/RFP stage, there is no team existing. A rudimentary estimation has to happen for the scope/size/tenure of the project. After this the programmers/PLs/designers/analysts join this project. This estimation happens based on person's experience.


This doesn't sound like a very practical situation to me. Although many organizations choose to work this way, in practice it proves to be a really poor way to work. My recommendation is to educate people on the implications of the choice to work this way, and my BRUF article should help to do that, and then let them know that there are better ways to work.

Just because something is the status quo in your organization, that doesn't mean it's a good way to work, nor does it mean that it's practical. If the person making the estimate was truly experienced they would know that working this way isn't a very good idea.

- Scott
Rajah Nagur
Ranch Hand

Joined: Nov 06, 2002
Posts: 239
Originally posted by Ilja Preuss :"

Well, if that is working well for you, there is no reason to change I guess.
Is it working well for you?
[QB]

Not always (My personal opininon). But I think folks in the IT world are used with this kind of System & they tend to find something good from it even if the project dooms. Human brain perceives what it wants.

Originally posted by Scott Ambler:
[QB]

This doesn't sound like a very practical situation to me. Although many organizations choose to work this way, in practice it proves to be a really poor way to work. My recommendation is to educate people on the implications of the choice to work this way, and my BRUF article should help to do that, and then let them know that there are better ways to work.

Just because something is the status quo in your organization, that doesn't mean it's a good way to work, nor does it mean that it's practical. If the person making the estimate was truly experienced they would know that working this way isn't a very good idea.

- Scott


I read the entire article. One question, can you please tell what are the assumptions you had thought of while writing this article?
(For e.g. The customer as well the team are both versatile in the technology they are handling etc)
Scott Ambler
author
Ranch Hand

Joined: Dec 12, 2003
Posts: 608
To make this approach work, you must:
1. Have regular access to your stakeholders, something that is ALWAYS possible ( see Overcoming Requirements Challenges ).
2. Have competent developers who are willing to work together.
3. Have competent stakeholders who are willing to work with the team.

Very reasonable assumptions, yet most teams struggle to meet them. I guess that my fundamental assumption is that you're willing to choose to succeed, which I have argued for years is a very difficult decision to make (people choose to fail because it's much easier to accomplish, then lie to themselves about why the failure occured).

- Scott
Rajah Nagur
Ranch Hand

Joined: Nov 06, 2002
Posts: 239
Originally posted by Scott Ambler:
I guess that my fundamental assumption is that you're willing to choose to succeed, which I have argued for years is a very difficult decision to make (people choose to fail because it's much easier to accomplish, then lie to themselves about why the failure occured).

- Scott


Very well-said Scott. There is lot more clarity now after reading "Overcoming Requirements Challenges". Thanks for your time
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Scott Ambler:
(people choose to fail because it's much easier to accomplish, then lie to themselves about why the failure occured).


I think another point is that people choose approaches not mainly on the basis of how likely it is to succeed, but on how well they know them. They'd rather fail in a way they are used to, than improve the likeliness to succeed by trying something new.

And in fact, in some sense trying something new *is* more risky - because when it fails, you can't say "it's not my failure - I have just done what everyone does."
[ July 11, 2006: Message edited by: Ilja Preuss ]
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Rajah Nagur:
Originally posted by Ilja Preuss :"

Well, if that is working well for you, there is no reason to change I guess.
Is it working well for you?



Not always (My personal opininon).


Mhh, and still from your previous post it sounded a little bit like you were defending that approach. Were you?


But I think folks in the IT world are used with this kind of System & they tend to find something good from it even if the project dooms. Human brain perceives what it wants.


There are a lot of folks who do that - and a lot who do not.
Rajah Nagur
Ranch Hand

Joined: Nov 06, 2002
Posts: 239
Mhh, and still from your previous post it sounded a little bit like you were defending that approach. Were you?


I meant,Sometimes it works and sometimes it doesn't.

I was defending in the sense because I was not thoroughly convinced *starting a project without any estimation* and also project always succeed when the developers/programmers contribute to the estimation.

I totally agree with your views and opinions. Things change rapidly in IT and Agile processes seem to make much more sense.

The fever is catching up...
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Rajah Nagur:
I was defending in the sense because I was not thoroughly convinced *starting a project without any estimation* and also project always succeed when the developers/programmers contribute to the estimation.


Well, and you shouldn't be convinced of *that*.

What I'm convinced of, and what I therefore tried to say, is that you should start a project with an estimate, but that that estimate will only be very rough whatever you do, so diving down into small details is unlikely to improve it much. Involving those people who later have to work to the estimate is much more likely to improve the results. That's all...
Yufeng Su
Greenhorn

Joined: Aug 01, 2006
Posts: 1
I think there is no definitely best methodologies for any project, no matter traditional methodlogy or agile, it depends on the scales of you project and the resources.

I think agile is suitable for the small or medium projects which the developing or maintaining process won't be too long. It counts much on the factors of humman, so the stabilities and the abilities of the humman resource is the most important for the delivery.
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Yufeng Su:
I think there is no definitely best methodologies for any project


Well, yes, of course.

no matter traditional methodlogy or agile, it depends on the scales of you project and the resources.


Well, what would have to be true for you to value

- processes and tools over people and interactions,
- comprehensive documentation over working software,
- contract negotiation over customer collaboration, and
- following a plan over responding to change

???

I think agile is suitable for the small or medium projects


I don't have any experience with large projects, but I do know some people who have a lot of experience with it and say that Agility is *especially* important for large projects.

which the developing or maintaining process won't be too long.


That's a strange statement, as Agile approaches put a strong emphasis on making development sustainable.

I'm currently working on a project that is under development and in production for around ten years, counting. Starting to introduce Agile practices five years ago has saved our asses.

It counts much on the factors of humman, so the stabilities and the abilities of the humman resource is the most important for the delivery.


Well, yes, of course. That's why Agile approaches put so much emphasis on humanity.
 
 
subject: Agile Fixed cost development?