Practical difficulties of Agile

[Vinayagam Kulandaivel]

What are the practical dificulties to implement Agile model in mission critical application software development.

Thanks
Vinayagam

[Beppe Catanese]

And when (if ever) will you NOT advice an agile approach in a given project/context?
Or there is always a possibility to adjust/tune/adapt your agile techniques to fit your projects?

Thanks
Beppe

[James Shore]

Hi Beppe,

I think a team with sufficient expertise can make agile development work with any project... assuming the other people on the project want to play along. I'm cheating a bit when I say that, though, because I expect that our hypothetical experts will use any tools necessary, including "non-agile" things like high-ceremony documentation, when appropriate. Given a sufficiently contrived scenario, our agile experts could create an "agile" waterfall method.

Ultimately, I think mastery means knowing how to tune and adapt the process. Experts have a better idea of where to start, too, but trying things and experimenting with improvements--even at the process level--is core to agile development. Tools like retrospectives help teams do so.

However, our book is written for people who aren't at that level of expertise yet, so it is not appropriate for all situations. We provide a list of prerequisites and recommendations in a section called "Is XP Right For Us?" that includes requirements such as management support, colocated teams, between four and 20 team members, and so forth. You can still use the ideas in the book, but they'll need to be adapted to your situation if you don't meet the prerequisites. We provide some ideas on how to do so.

Regards,
Jim

[Stan James]

There can be tremendous political challenges. With full blown Scrum and other agile methods, job descriptions and responsibilities change and disappear. Some shops might have to involve HR for job titles, career paths and compensation incentives. Once you have the development team organized, you need customers who will participate in iteration planning and testing, officers who can handle the absence of a long QA phase, etc.

I'm interested in the authors' experiences in these areas. For a proof of concept I imagine you can organize a small AD team any way you like, but for permanent change, some of these issues surely come into play, no?

[James Shore]

Hi Stan,

Absolutely, and I should have stated that more clearly in my original post. Although I think agile methods can be customized for any (or nearly any) technical situation, the real challenge in using agile methods is getting the people to agree and cooperate.

As a consultant, I work with a lot of organizations that are going through this transition. I'm sorry to say there's no magic way to convince people to try agile development. I take a two-pronged approach: first, I'm very non-threatening about it. I make sure that everybody knows that agile development is their choice to make and that we'll evaluate our progress at a specific date (usually several months in the future). If the team decides it isn't working, we'll stop.

Second, I only work with teams and organizations that are genuinely willing to try an agile approach. That doesn't mean that everybody wants to do agile, but it requires that people not be actively fighting it. If there are people fighting it, I figure out a way to make sure they don't have to try it, even if that means we don't do anything agile at all.

I know this doesn't help people who are trying to change an organization from within. My Change Diary might be helpful, and Rising & Lynn-Mann's Fearless Change is another good resource.

[Vinayagam Kulandaivel]

Folks,

Thanks for your time.

It's really very dificult to define mile stones in agile model.
How to over come this do you have any suggestion Jim.

Thanks & Regards
Vinayagam

[Ilja Preuss]

Originally posted by Vinayagam Kulandaivel:

It's really very dificult to define mile stones in agile model.

Uh, why would that be difficult?

[Vinayagam Kulandaivel]

Hi Ilja,

Here are the mile stones

1. Requirement analysis

2. Design

3. Construction

4. Testing

5. Implementation/Delivery

6. Maintenance/Management

Lets assume the the requirement is freeze/clear.

Even we have done the design (I hope the design will not get freeze in first time itself)

Now we have started construction using some iterations.
Due to some reason my design should have to be updated/redesigned.
Also it requires testing too. As far as my knowledge the 2, 3 and 4 mile stones will be relative. Am i making sence?

Thanks & Regards
Vinayagam

[James Shore]

Hi Vinayagam,

The milestones you've defined are phase-based milestones that make little sense in an agile environment. Remember, analysis, design, coding, and testing happen every iteration in an agile project. (And the iterations are less than a month long.) In an XP project, they actually happen simultaneously.

The milestones you've described are specific to phase-based models. Agile teams use different milestones that are based on features delivered to customers.

[Ilja Preuss]

I would even argue that those things aren't real milestones, they are more like phantom milestones.

What's the purpose of a milestone? As far as I can tell, it's to know when you have reached it how much of the project is done and how much is left to be done.

Now, how do you know how much of the project is done when you're done with design? You can't tell, because you don't even know whether the design is correct. Only while implementing the design is it really validated. And nothing in the design will tell you how long it will take to implement the design, what road blocks you will encounter etc.

Do you know how much of the project is done when you are done with construction? Not really, because you haven't done any testing yet, so you don't know the quality of the implementation, so you don't know how much of the implementation will need to be redone.

Etc. pp.

[Vinayagam Kulandaivel]

Great Post, Thanks Jim & Ilja..

Regards
Vinayagam

[Shiang Wang]

In a sprint of two-week or three-week iteration, we usually end up with QA struggling with finishing testing in time. Do you have any suggestion how we get around it?

The cause of the problem is that we tend to check in the code close to the end of the sprint due to the complexity of the implementation or testability of the code change. There are some talks about separating QA sprint from Development sprint, what are your thoughts on this?

[James Shore]

Hi Shiang,

You're describing a common problem that's a result of using sequential phases in short iterations. XP's approach of using simultaneous phases works better; see my post on how QA fits in for more.

I strongly recommend against separating a QA sprint from the development sprint; you'll end up with bottlenecks and rework. The point of the sprint is to have known-good, completed, ready to ship code at the end of the sprint. This reduces risk and improves quality. Separating QA out will lead to all sorts of problems: backlogs of incomplete work, unexpected rework, large bug queues, and more.

Learn how to use XP-style simultaneous phases instead. It works much better.
[ November 01, 2007: Message edited by: James Shore ]

[Ilja Preuss]

In short, if QA isn't finished at the end of the sprint, they need to start *sooner*, not later.

I don't understand the part about complexity, because continuous integration *reduces* complexity, in my experience.

Anyway, there are ways for QA to start writing their tests without having access to the code. There are even teams out there that get all the tests from QA at the *start* of the iteration.

[Shiang Wang]

Jim,

Thanks for the suggestion.

The biggest challenge we have is how to execute the acceptance tests we setup with QA in user story time. It looks like the process we have put in place needs some change.

[Shiang Wang]

Ilja,

What I mean is some of the tasks we have on hand can't be easily broken into incremental check-in. If we even do, those changes are not testable to the QA, usually the backend change in order to support UI.

[Vinayagam Kulandaivel]

Folks,

Any guidelines or suggestions for Function Point Analysis and Cost Estimation in an AGILE development environment.

Thanks & Regards
Vinayagam.

[James Shore]

Hi Vinayagam,

The correct answer to your question is "what are you trying to accomplish?" but I'm tired so I'll just refer you to my recent scheduling post.

[Ilja Preuss]

Originally posted by Shiang Wang:
What I mean is some of the tasks we have on hand can't be easily broken into incremental check-in.

It's a skill that can be learned. Today I'm quite positive that I can break down *any* programming task so that I can check in at least several times a day.

If we even do, those changes are not testable to the QA, usually the backend change in order to support UI.

Using an Agile approach, it is *not* QAs responsibility to test features once they are testable. It is instead their responsibility to prepare tests in advance, so that once a feature is finished, testing it is a matter of pressing a single button (or often not even that, using an integration server).

[Shiang Wang]

We are getting there. Currently we still rely QA's hands clicking on those buttons to carry out the test plans. That is what kills us at the end of the sprint.

[Stan James]

Ilja:

Using an Agile approach, it is *not* QAs responsibility to test features once they are testable. It is instead their responsibility to prepare tests in advance, so that once a feature is finished, testing it is a matter of pressing a single button (or often not even that, using an integration server).

Wow, I am way too bound by the "current state" of our organization. That's just way cool.

James Bach says automated tests almost never find bugs, and if you code to the tests they never will. Of course your approach leaves QA more time to do the exploratory testing Bach likes.
[ November 07, 2007: Message edited by: Stan James ]

[Ilja Preuss]

Originally posted by Stan James:
James Bach says automated tests almost never find bugs, and if you code to the tests they never will.

Well, that doesn't match my experience at all.

One thing to keep in mind is that automated tests are even more important for an Agile team, because of all the refactoring that needs to happen.

Of course your approach leaves QA more time to do the exploratory testing Bach likes.

Very true!

[Jeff Langr]

Originally posted by Vinayagam Kulandaivel:
Any guidelines or suggestions for Function Point Analysis and Cost Estimation in an AGILE development environment.

I don't normally respond so strongly about a practice, but I'm convinced that function points are a waste of effort. My even more cynical reaction is that they largely exist to sustain consulting revenue.

Yes, they certainly can end up being accurate at times. They might also represent an OK way to compare the estimation quality from one project to another, but there is little to no value to incorporating function points on an agile project. Function points are fairly complex to derive, and you do need an "expert" in order to do well with them.

I've seen as good or better results from agile estimating and planning techniques, which come more cheaply, are far less onerous, and anybody can quickly learn how to do them without hiring an overpriced consultant (or "software economist").

Jeff

[Ilja Preuss]

Originally posted by Jeff Langr:
Yes, they certainly can end up being accurate at times.

I don't remember where, but recently I read about three different consultants coming up with function point estimates for the same project that differed by a factor of 3.

I've seen as good or better results from agile estimating and planning techniques, which come more cheaply, are far less onerous, and anybody can quickly learn how to do them without hiring an overpriced consultant (or "software economist").

Our team recently adopted the Planning Poker practice, and for us it seems to work quite well.

[Scott Ambler]

A few thoughts:
1. Up front, detailed estimating practices such as Function Points (or COCOMO, or ...) sound great in theory by actually motivate some very risky development practices as a result. See http://www.ddj.com/architect/199001126?cid=Ambysoft for some thoughts.
2. I suspect that James Bach is being quoted out of context. The real issue is between confirmatory testing, which TDD is all about, and investigative testing, which TDD by definition doesn't address. The biggest bang for your buck is often in investigative testing although it assumes that you've nailed the straightforward stuff that confirmatory testing addresses. See http://www.ddj.com/development-tools/196603549?cid=Ambysoft

- Scott