When is the best time for some analysis and design on an agile project? - on Learning Agile

[charlsy chuks]

I am currently working on a team and I am pretty sure we are adopting agile strategies even though we are not following it strictly.
We basically design one module at a time and implement partly then move to another. At the points where one interact with another we revisit the previous ones.
When the client says a new feature should be added or asks for modification of an existing feature, it could involve heavy change in database schema, not to mention code.

I often wonder if this is what agile is really about because it could be stressful making changes and undoing stuff that took time to put in place.

Hence my question 'When is the best time for some analysis and design on an agile project?' Thanks.

[Junilu Lacar]

charlsy chuks wrote:I often wonder if this is what agile is really about because it could be stressful making changes and undoing stuff that took time to put in place.

Hence my question 'When is the best time for some analysis and design on an agile project?' Thanks.

Although there really aren't any good short answers, I'll offer two: "Whenever you need to." and "All the time."

Of course, there are many facets to the long answers and entire books have been written about agile design. Concepts that you might want to look into include emergent design, clean code, refactoring, design principles like SOLID, DRY and SLAP, continuous integration, automated testing, and Test-Driven Development. When done properly, all these contribute to agility and the team's ability to effectively evolve the software as you learn more about the system that you are developing.

[Andrew Stellman]

The best for analysis and design is the last responsible moment. This is one of the ideas that really differentiates agile teams from more traditional ones, and it's central to having an agile mindset.

Doing analysis and design last responsible moment means making only those design decisions that need to be made right now, and putting off any others that can responsibly be put off. It's a pretty simple idea, but it has a huge impact on how the team designs their software. Effective XP teams are especially good at making design and analysis decisions at the last responsible moment, and it helps them keep their code simple.

If you're used to working on a team that follows a waterfall process this may seem very strange, because you're used to having a separate project phase for design, and all of the design work happens in that phase. Doing some design and analysis early in the project but leaving other design decisions until later, even late in implementation, often feels very "loosey-goosey" or haphazard to someone used to the strict project structure of a waterfall process. And if you're in a company with a culture where making mistakes is not tolerated or even career-ending, making design and analysis decisions early and getting them reviewed by absolutely everyone in the project feels much better than leaving them for the last responsible moment because you get the CYA of everyone seeing your design early and signing off on it, so if mistakes were made, it's not exclusively your fault. But the flip side is that you're tied to decisions that could have been made later, and this makes you and your team much less flexible.

Agile teams take a similar latest responsible moment approach to planning, too, which lets then have much more simple and flexible plans.

By the way, one key to making design and analysis (or planning) decisions at the latest responsible moment is getting the whole team involved in the discussion, and creating a collaborative workplace where everyone who has something positive to add to the discussion feels comfortable doing so.

[Junilu Lacar]

Andrew Stellman wrote:The best for analysis and design is the last responsible moment.
...
Doing analysis and design [at the] last responsible moment means making only those design decisions that need to be made right now, and putting off any others that can responsibly be put off. It's a pretty simple idea, but it has a huge impact on how the team designs their software.

It's simple in much the same way that Refactoring and Test-Driven Development are simple. Unfortunately, it's far more difficult in practice. It takes quite a bit of experience and skill to know exactly which decisions can be put off and which ones need to be made right now. An even more difficult thing to discern is just how much is just enough. The issue I have with "the last responsible moment" is that to many, it implies a single point in time when really the idea is to defer certain detailed aspects of the design decision while making decisions about the more fundamental aspects up front. Sometimes, the "last responsible moment" is really much sooner than many inexperienced developers can discern, hence you get a lot of "well, it's too late to change that now" situations. That's why I've taken to saying that design should be done all throughout the development process and since you can't do design without some level of analysis, you do that all throughout as well.

Take for example the oft-cited example of Fitnesse: They chose to defer the decision on using a database. This turned out to be a good choice on that project because they were able to defer the decision *away* altogether. For many other projects, the decision to use a database has to come up front and often the choice of which particular database system as well. Are there certain things about using a database you can defer? Absolutely. But it takes skill and experience to know what those things are, when to decide, and how to position yourself so that you can back out that decision and change direction.

[Junilu Lacar]

I think that's really where many conversations about the "last responsible moment" fall short: there's often not enough mention of or thought put into the things you have to do to position yourself so that you can effectively defer certain decisions. Automated testing is big. Clean code and well-factored designs are big. Collaboration and a shared understanding between the development team and the customer is big. Continuous Integration and Delivery is big. But these are often left unsaid or implied and a lot of teams miss that. All they hear is "Defer the decision until you can't defer it any more" and then blindly keep going the way they're going, often painting themselves into a corner.

[charlsy chuks]

Great replies Andrew and Junilu,

However, if I got you guys right it's not a question that has a fixed answer and could vary from project to project or even from team to team. I came to the conclusion that some tools and technologies lend themselves more to Agile style of development than others. I don't know if it is okay to recommend some here.

Concepts that you might want to look into include emergent design, clean code, re-factoring, design principles like SOLID, DRY and SLAP, continuous integration, automated testing, and Test-Driven Development. When done properly, all these contribute to agility and the team's ability to effectively evolve the software as you learn more about the system that you are developing.

I was just wondering that given the nature of agile development, which of these principles fits BEST into the agile framework?

The best for analysis and design is the last responsible moment. This is one of the ideas that really differentiates agile teams from more traditional ones, and it's central to having an agile mindset.
.....Agile teams take a similar latest responsible moment approach to planning, too, which lets then have much more simple and flexible plans.

I just wanted to be sure; was that a typo or are there really differences between last and latest?

Take for example the oft-cited example of Fitnesse: They chose to defer the decision on using a database. This turned out to be a good choice on that project because they were able to defer the decision *away* altogether.

I wonder if any 'serious' development can be done 'without' a database either relational or object kind like you cited in Fitnesse. A risk I sense here is that in doing bits of the system at different times, after a while especially without adequate comments and documentation, one forgets how the entire system fits in and works hand-in-hand. By that I mean how the different modules should
interact. Hence those who say a huge part of the design should be done upfront with about 90% clarity of the BIG picture of the system may have a point. Do you agree?

[Junilu Lacar]

charlsy chuks wrote:
I wonder if any 'serious' development can be done 'without' a database either relational or object kind like you cited in Fitnesse. A risk I sense here is that in doing bits of the system at different times, after a while especially without adequate comments and documentation, one forgets how the entire system fits in and works hand-in-hand. By that I mean how the different modules should
interact. Hence those who say a huge part of the design should be done upfront with about 90% clarity of the BIG picture of the system may have a point. Do you agree?

I think this supports my assertion that there is a lot of confusion as how much is just enough. "Huge" can take on different meanings. To me, something that is architecturally significant is a "huge" decision although I can position the code and tests so that I can still make just enough decisions about architecture and defer other details for later. For others, "huge" means they will spend three weeks drawing and documenting the most minute details; this seldom results in the kind of decisions or decision making we're talking about.

BTW, I can't imagine that development of Fitnesse was any less "serious" than any other typical non-trivial project. Sure, it probably doesn't do as much as your everyday enterprise-level application but the development effort that went into it in the time that it took them to ship it was probably the same as any other project of similar size.

[Junilu Lacar]

charlsy chuks wrote:

Concepts that you might want to look into include emergent design, clean code, re-factoring, design principles like SOLID, DRY and SLAP, continuous integration, automated testing, and Test-Driven Development. When done properly, all these contribute to agility and the team's ability to effectively evolve the software as you learn more about the system that you are developing.

I was just wondering that given the nature of agile development, which of these principles fits BEST into the agile framework?

And exactly what "nature" is that? To me, all these help you develop with more agility, each contributing in their own way but taken together, they create a synergy that makes the whole larger than the sum of the individual parts.

[charlsy chuks]

Well said Junilu,
When I said 'huge' I meant having a thorough understanding of the system by quizzing the client, potential users and other stakeholders plus analyzing the whole system to be automated. But I totally understand your point of view.

The challenge I think most teams have is they implement some aspects of agile and leave others which are important. Like I know I have not really tried automated testing before and probably came across SLAP for the first time here. However, I have attempted using DRY by putting code thats called in multiple places into methods so if I need to change I simply update the method once. I can just imagine having a database where a foreign key relationship exists and lots of code has been written around that relationship only to be changed at the last minute because of a new update to functional requirements.

If the system were database access agnostic, it would only require changing the data access queries which in itself could be pretty difficult.

Truly that synergy is what we all need.