What happens when the client doesn't know what they want?

[leo donahue]

I am recent grad. In school we learned that systems analysis and design is essential to helping the customer know what they want so we, as the developers, don't spend alot of time giving the customer what they don't want.
Don't get me wrong, after reading the definition of XP, I realized that I was pretty much already coding in XP style and didn't know it. Most of the time I try to write a small program that does exactly what I want a particular mehtod or function to do. Then later, if my boss wants the feature improved, they'll let me know.
So my question is, what happens when your boss asks you for some kind of improvement that will drastically change the face of your program? By not planning for change or growth, how will I continue to benefit from XP?

[Richard Jensen]

Originally posted by leo donahue:
So my question is, what happens when your boss asks you for some kind of improvement that will drastically change the face of your program? By not planning for change or growth, how will I continue to benefit from XP?

The premise of Extreme Programming is embrace change. You don't necessarily plan for it, but you recognize that it is going to happen and therefore write your code so that it:

Runs all the tests.

Has no duplicated logic.

States every intention important to the programmers.

Has the fewest possible classes and methods.

This list is from Kent Beck's book: "Extreme Programming Explained"
Such code is malleable as well as maintainable. As new requirements come along you have a well-prepared foundation that is adaptable.
[ May 14, 2003: Message edited by: Richard Jensen ]

[Junilu Lacar]

Originally posted by leo donahue:
So my question is, what happens when your boss asks you for some kind of improvement that will drastically change the face of your program? By not planning for change or growth, how will I continue to benefit from XP?

After reading the XP book, did think that XP is supposed to impede change and growth? On the contrary, unit tests and refactoring work together to make it easier to change and grow your program. Many misconceptions about XP are caused by mismatching an element of XP and a goal. In your case, you are probably taking "You Ain't Gonna Need It" and saying "Oh, does that mean I won't plan for change and growth?" The answer is: No. But by writing unit tests and refactoring your code so that you always have a well-structured design, you will be better able to introduce changes and growth. The only difference is that you won't have to dwell on trying to see into the future and anticipate exactly what kind of changes and growth will be needed.
I have experienced this many times. As long as I follow basic design principles and keep my code and design "clean", I am often rewarded by the ease with which I can make unplanned-for changes to the code in the future.

[Mark Herschberg]

This is one of my major criticisms of XP. XP correctly recongizes change is inherent in software projects. However, XP is only good at embracing small to medium sized change. I am skeptical of XP handling major architectural changes.
Now if you're getting significant architetcural changes, there's a serious problem (likely outside of engineering). Presumably, even if you don't have all the stories upfront, you've got enough to them to make reasonable judgements about how to design the overall system. As requirements/stories change you modify your code appropriately. Still, occasionally external factors (economic, marking, catastrophy, political, management, competitive, etc) do result in the need for such a radical change. I don't think XP is well designed for that. Granted, most projects will never face this, but if your project might, XP may not be the right way to go.
On the other hand, I'm also a big fan of, write once to throw it away. If you're facing a really really major of a change, you may just want to scrap it and start again anyway.
--Mark

[Lasse Koskela]

Originally posted by Mark Herschberg:
On the other hand, I'm also a big fan of, write once to throw it away. If you're facing a really really major of a change, you may just want to scrap it and start again anyway.

I believe nearly all developers have been in a situation where they'd give anything, anything to get permission for a rewrite... It's just those darn managers again complaining about costs and insist on extensive reuse (while 2 months earlier they were strongly against reuse using the YAGNI buzzword they heard from the techies:)

[Junilu Lacar]

Originally posted by Mark Herschberg:
This is one of my major criticisms of XP. XP correctly recongizes change is inherent in software projects. However, XP is only good at embracing small to medium sized change. I am skeptical of XP handling major architectural changes.

I do not see how using XP would limit you to small and medium-sized changes in architecture. In his most recent book, Martin Fowler broadly defines "architecture" as those design decisions that are hard to change. If we go by this definition of architecture, then it doesn't really matter if you use XP or not.
But, overall design does affect your ability to change the architecture. The better the overall design, the easier it is to change. Since XP practices promote good design, wouldn't it follow that it would also make it easier to make architectural changes?
And even if, for the sake of argument, you were right about XP being able to handle only small to medium-sized changes, then couldn't a large change be effected by iteratively making small to medium-sized changes?

[Richard Jensen]

Originally posted by Mark Herschberg:
... I am skeptical of XP handling major architectural changes.
...
--Mark

This question/issue comes up repeatedly on the various groups where I lurk. The posters who have the most expererience with XP are the ones with the least concerns in this area. I don't know how many times I've seen one poster talk about using a database and getting responses along the lines of "Is a database the simplest thing that could possibly work?" and relating experiences (yes, anecdotal but actual) of growing their system using XP practices and never needing the database they originally envisioned. Or, when they DID need it, their access layer was easily adapted because of the Once and Only Once and merciless refactoring that had occurred to that point.

[Frank Carver]

One more thought in the pot.
In my experience, adopting a "test-driven", "you aren't gonna need it", "merciless refactoring" approach almost always produces software that is both smaller and cheaper than attempting to design for future change.
Let's say for example, or XP team can produce a well-factored but single-use application for $5000, and the BDUF team produces a larger system with some sort of designed-in hooks and options for $10000.
Now, imagine a major "architectural" change comes along.
In practice, well-factored designs often have a surprisingly high level of reusability, but even if all the XP solution is no longer useful, a replacement will presumably cost about the same as the first one ($5000, making a total of $10000).
By definition, this "architectural" change is major and unexpected, so we must assume that a large proportion of the BDUF solution will need to be changed, too. let's be generous and say that only half the system needs to be replaced, and a cost of $5000, making a total of $15000 for the whole system.
Sure, my figures are disputable, they are only based on a gut feeling from my own experience, but I hope the logic is reasonable. If you build light, fast, cheap and only what you need now, there's less to throw away and rebuild if/when that "architectural" change comes along.
I think this is one of the reasons that experienced XP practitioners don't seem to worry about this. Continual refactoring means that no code is sacred. Replacing big swathes of code is no big deal.

[Lasse Koskela]

Sidenote: I really don't know how this relates to the thread's title anymore, but...
How does XP and reuse go together if at all?

[Mark Herschberg]

Originally posted by Junilu Lacar:
But, overall design does affect your ability to change the architecture. The better the overall design, the easier it is to change.

This strikes me as one of those truisms like "more people will get the project done faster." Remember, these changes aren't always rational; pointy haired bosses do take left turns down unpaved roads ("what idiot would want to design a system this way?") in the middle of the project.

Originally posted by Junilu Lacar:

I do not see how using XP would limit you to small and medium-sized changes in architecture. In his most recent book, Martin Fowler broadly defines "architecture" as those design decisions that are hard to change. If we go by this definition of architecture, then it doesn't really matter if you use XP or not.
...
And even if, for the sake of argument, you were right about XP being able to handle only small to medium-sized changes, then couldn't a large change be effected by iteratively making small to medium-sized changes?

That's the crux of my argument. I am skeptical of a 10 person team all pulling in the same direction.
In XP, everyone kinda does his own thing. That is you grab a card, write some code, which may go anywhere in the system, and meet your objectives. Your implementation may "step on the toes" of a co-worker in that it makes his next task a little harder. No problem, he refactors your code to make it easier. This is the same way open source projects work. (For the record, technically everyone on a project does his own thing, but traditionally everyone has his or her own backyard to play in.) The process isn't totally "efficent" but when code does get misaligned it can be quickly and easily modified.
With architecture, I'm not sure the same is true. I fear that if the 10 people are not all envisioning the same architecture, their misalignments in objectives will cause trouble. Architecture often tends to be something with more global implications, so imperfections get magnified. I believe the reasons why design by committee does not work come into play here.
We know there's always more then one way to do something. XP promotes fluid code. If the way something was originally done isn't quite right for the next objective, we change how it was done. On a small scale this works. Most XP tasks are on the order of a few man-hours to a few man-days. Archtectural changes are on the order of man-weeks to man-months. I fear that during such changes people will find the rug pulled out from under them!
More concretly, I think architectural design and changes need more formal planning then other design decisions. For these points in the code, I would suggest less of an XP aspect.
Let me put it another way. I don't know of any XP projects where everyone jumped right in; i.e. as soon as the last case was done, everyone started coding. Some time was spent considering system design. I think this time must be spent again.
Does this mean XP doesn't work? No, just that you need to demphasize certain aspects of XP during this time. I also believe that a good team with good technical leadership could do this, I just wouldn't recommend it to most teams.
--Mark

[Junilu Lacar]

Originally posted by Mark Herschberg:
In XP, everyone kinda does his own thing. That is you grab a card, write some code, which may go anywhere in the system, and meet your objectives. Your implementation may "step on the toes" of a co-worker in that it makes his next task a little harder. No problem, he refactors your code to make it easier.

On the contrary, XP promotes communication and a shared vision. "Toe-stepping" is caused by a lack of communication. XP promotes constant communication both through increased developer interaction and clarity of code/intent.

This is the same way open source projects work. (For the record, technically everyone on a project does his own thing, but traditionally everyone has his or her own backyard to play in.) The process isn't totally "efficent" but when code does get misaligned it can be quickly and easily modified.

I doubt that XP practices can be effectively applied to open-source projects where the team members are not co-located.

Most XP tasks are on the order of a few man-hours to a few man-days. Archtectural changes are on the order of man-weeks to man-months. I fear that during such changes people will find the rug pulled out from under them!
[..]
More concretly, I think architectural design and changes need more formal planning then other design decisions. For these points in the code, I would suggest less of an XP aspect.

I believe the "spike solution" is one way to handle these concerns in XP. You try to quickly develop the core solution to see if you're on the right track.

This strikes me as one of those truisms like "more people will get the project done faster."

I thought this was a fallacy
Junilu

[Ilja Preuss]

Originally posted by Mark Herschberg:
Remember, these changes aren't always rational; pointy haired bosses do take left turns down unpaved roads ("what idiot would want to design a system this way?") in the middle of the project.

How would you want to anticipate those changes before they happen?

That's the crux of my argument. I am skeptical of a 10 person team all pulling in the same direction.
In XP, everyone kinda does his own thing.

That would certainly be bad. And I don't think it's true.

That is you grab a card,

Remember, this card was discussed in both the release and iteration planning meeting by the whole team. It needed to be estimated, so you probably needed to have *some* rough idea of how to implement it. It's certainly not uncommon to hear conversations like "oh, for that we only need to extend the Foobilator and sqoze the Quadrumon. That's a '1'." in the planning phase.

write some code, which may go anywhere in the system, and meet your objectives.

And it needs to conform to the system methaphor, as defined by the team. And all of your pairing partners need to agree on where to put it. You might even do a quick design session on the white board.

Your implementation may "step on the toes" of a co-worker in that it makes his next task a little harder. No problem, he refactors your code to make it easier.

"Mark, I see (from CVS) that you have worked last on the Gramboon. I have some problems using it for my task, do you have time to help me?"

This is the same way open source projects work. (For the record, technically everyone on a project does his own thing, but traditionally everyone has his or her own backyard to play in.) The process isn't totally "efficent" but when code does get misaligned it can be quickly and easily modified.

I don't think it's quite the same.

With architecture, I'm not sure the same is true. I fear that if the 10 people are not all envisioning the same architecture, their misalignments in objectives will cause trouble. Architecture often tends to be something with more global implications, so imperfections get magnified. I believe the reasons why design by committee does not work come into play here.

Yes. I don't think that is what is happening in a typical XP team.

We know there's always more then one way to do something. XP promotes fluid code. If the way something was originally done isn't quite right for the next objective, we change how it was done. On a small scale this works. Most XP tasks are on the order of a few man-hours to a few man-days. Archtectural changes are on the order of man-weeks to man-months. I fear that during such changes people will find the rug pulled out from under them!

Yes, we all know this fear. But it seems as if incorporating small features typically don't need big architectural changes, only small ones. It seems as if big architectural changes can be broken up in small architectural changes, in a way that each such change provides business value.
There may be situations where this doesn't work. It's not clear to me if they happen frequently enough to let them dictate my fears.

Let me put it another way. I don't know of any XP projects where everyone jumped right in; i.e. as soon as the last case was done, everyone started coding. Some time was spent considering system design. I think this time must be spent again.

Mhh, I don't think a good XP team *ever stops* to think about these issues.

[Ilja Preuss]

Originally posted by Lasse Koskela:
How does XP and reuse go together if at all?

Well, as an XP team doesn't tolerate duplication, they have to maximize reuse, so it seems to me.
I somehow have the feeling that this doesn't really answer your question, though. Perhaps you should try again...

[Lasse Koskela]

Originally posted by Ilja Preuss:
Well, as an XP team doesn't tolerate duplication, they have to maximize reuse, so it seems to me.
I somehow have the feeling that this doesn't really answer your question, though. Perhaps you should try again...

You're correct. That wasn't exactly what I was looking for...
What I meant with "reuse" was organizational, Experience Factory type-of reuse. If the code produced by XP teams conforms to YAGNI and the-simplest-thing-that-could-possibly-work, how would the components adapt for generalization (preparing for reuse by other projects)? Then again, if the XP teams are refactoring mercilessly, aren't we digging the ground from component-based reuse? As far as I know, successful reuse programs are pretty much based on strictly controlled interface evolution.

[Frank Carver]

In my experience reuse in an extreme shop works like reuse in the open source community. If you see something you like, use it. It can be as little as a few lines of code or as large as a whole application.
Because reuse typically happens at the source-code level rather than reusing black box components, it is assumed that they will probably diverge, but that's OK, you've already had the benefits of the reuse.

[Lasse Koskela]

Originally posted by Frank Carver:
Because reuse typically happens at the source-code level rather than reusing black box components...

So, either the team members need to know the code of all projects done within 3 years or they can wake up from the wet dream of reusing existing software...
I can see the cut'n'paste scope of reuse with XP. I was just wondering whether "real", component-level reuse was possible (or feasible to be more accurate) with XP.

[Eric Burke]

Originally posted by Lasse Koskela:
How does XP and reuse go together if at all?

For reusability within a single application, I think pair programming helps in this respect because you spread knowledge. Programmers are more likely to be aware of code that has already been written because the communication among team members is so much better.
Consider what happens when you have a "cowboy" who always works in isolation from the rest of the team. I think the likelihood of someone reusing his code is diminished because he does not communicate. Working in isolation breeds duplication.
At an enterprise level, I think unit tests foster reusability. You'll always have the problem of finding the reusable code in the first place...I don't know that XP addresses that in any way. But once you have found the code, however, it is a heck of a lot easier to understand and trust the software if it is well tested.
If someone gives me a big blob of untested code and says "reuse this...it is the company standard", I'm skeptical. I just don't trust it unless it comes with tests that I can study.

[Mark Herschberg]

Originally posted by Junilu Lacar:
On the contrary, XP promotes communication and a shared vision. "Toe-stepping" is caused by a lack of communication. XP promotes constant communication both through increased developer interaction and clarity of code/intent.

I didn't phrase that well. By "toe-stepping" I meant that I optimize locally for my needs, and might not design well for your needs when you interact with my code tomorrow (hence the quotes).

Originally posted by Junilu Lacar:
I doubt that XP practices can be effectively applied to open-source projects where the team members are not co-located.

I would agree. Just to be clear, I never said open source is XP, but rather that XP and open source use common methods. Specifically, both groups have an open set of features/bugs and people tackle them somewhat independently.

Originally posted by Junilu Lacar:
I believe the "spike solution" is one way to handle these concerns in XP. You try to quickly develop the core solution to see if you're on the right track.

My impression was that the spike solution was not to be more then a few days. Besdies, right track still may not address many important issues.

Originally posted by Junilu Lacar:
I thought this was a fallacy

By now its well understood, but its one of those things that seems plausible enough at first. Another would be "open offices are good for communication." Seems to make sense, but turns out to be wrong.

Originally posted by Ilja Preuss:
How would you want to anticipate those changes before they happen?

How would you want to? Any way that's cost effective. :-p But the point is that these aren't predictable changes. (Is that really what you meant to ask?)

Originally posted by Ilja Preuss:
Mhh, I don't think a good XP team *ever stops* to think about these issues.

Now I'm confused, because this doesn't seem to go with your other comment:

Originally posted by Ilja Preuss:
Remember, this card was discussed...

So when does a team stop and discuss the issue? (Maybe we're using diffent definitions of "stop.")
More generally, I guess this is an issue of splitting hairs. Can XP do things well? Yes. My experience with people and skillsets is that their shortcomings won't be overcome by the XP model under the circumstances I described earlier.
--Mark

[Lasse Koskela]

Care to explain this to me, Leo. I don't follow.
How is this related to the snippet you quoted from my earlier post?

If we XP a process, then shouldn't that process be applicable to almost every business setting?

Who gave you the idea that processes are one-size-fits-all? It has been repeatedly stated that certain types of processes suit better to certain types of projects. One-size-fits-all is rarely true; in IT as well as other contexts.

[Frank Carver]

Lasse Koskela wrote: I can see the cut'n'paste scope of reuse with XP. I was just wondering whether "real", component-level reuse was possible (or feasible to be more accurate) with XP.
I think we need to work out what we want as the benefits of reuse. Just as with a lot of other things that we do in the name of software, reuse should be a means to an end, not an end in itself.
If by reusing an existing peice of software I can produce something which someone actually wants, but do it more cheaply, quickly or reliably, then the reuse has been beneficial.
If I spend two weeks evaluating alternative black-box components, or browsing source archives, then a week configuring and tweaking what I have found, when I could have written a perfectly serviceable local version in a week, the reuse has not been beneficial.
To reuse code at a source-code level doesn't mean that you have to know all the code in the world personally, just like reusing at a blck-box level doesn't mean that you have to know all the components in the world.
XP is essentially an iconoclastic process. XP encourages you to think of everything you do in terms of real costs and benefits to the customer, and never do things "just because". Most developers I have encountered seem to find source-level reuse easier, simpler and more controllable than black-box component reuse. If it gets the job done, do it.

[Ilja Preuss]

Originally posted by Mark Herschberg:
I didn't phrase that well. By "toe-stepping" I meant that I optimize locally for my needs, and might not design well for your needs when you interact with my code tomorrow (hence the quotes).

That is how it works out even if I am working solo - the code I optimize for todays needs isn't optimized for tomorrows. That's intentional, as I *can't fully know* about tomorrow needs. XPs solution is not to speculate about tomorrows needs, but to hold the code flexible and extensible *in general*.
You might certainly be sceptical about how well that works out. For me, it works *very* well...

people tackle them somewhat independently

Sorry, I can't imagine a process entailing closer collaboration than XP.

quote:
--------------------------------------------------------------------------------
Originally posted by Ilja Preuss:
How would you want to anticipate those changes before they happen?
--------------------------------------------------------------------------------
How would you want to? Any way that's cost effective. :-p But the point is that these aren't predictable changes. (Is that really what you meant to ask?)

What I meant to ask was: How would you want to anticipate unpredictable changes?
Here is what I wanted to do:
- hold the code clean (well decoupled, high cohesion, no duplication, no bloat)
- have extensible suites of tests, so that I don't break something if I have to make changes
- have as many from the team know as much about the system as possible

quote:
--------------------------------------------------------------------------------
Originally posted by Ilja Preuss:
Mhh, I don't think a good XP team *ever stops* to think about these issues.
--------------------------------------------------------------------------------
Now I'm confused, because this doesn't seem to go with your other comment:

quote:
--------------------------------------------------------------------------------
Originally posted by Ilja Preuss:
Remember, this card was discussed...
--------------------------------------------------------------------------------
So when does a team stop and discuss the issue?

They pause the discussion when they feel they can estimate the story.
They continue the discussion when they need to split the story up into task in the iteration planning meeting. They pause it when they feel they can estimate the tasks.
They continue the discussion when pairs form to implement the task. They pause it when they feel they know on which code they need to work.
They continue the discussion when they need to decide what test to write. They stop the discussion when they refactored the code to the simplest form they can imagine.

More generally, I guess this is an issue of splitting hairs.

Mhh, it didn't occure to me that this was the case...

My experience with people and skillsets is that their shortcomings won't be overcome by the XP model under the circumstances I described earlier.

Mhh, I wonder what might be a good alternative to giving people lots and lots of feedback on how they do...

[Ilja Preuss]

Originally posted by Eric Burke:
You'll always have the problem of finding the reusable code in the first place...I don't know that XP addresses that in any way.

It doesn't, it's outside its scope. I guess you could do anything you would do with any other process...

But once you have found the code, however, it is a heck of a lot easier to understand and trust the software if it is well tested.

It is also a heck of a lot easier to adjust it to your needs without breaking existing clients.

[Ilja Preuss]

Originally posted by Lasse Koskela:
If the code produced by XP teams conforms to YAGNI and the-simplest-thing-that-could-possibly-work, how would the components adapt for generalization (preparing for reuse by other projects)?

Typically, code produced by XP teams already is very well decoupled. That is because the code already is "reused" by the unit tests.
At work, we do have two rather distinct product lines, sharing a common library module. Whenever we write classes of which we think "hell, that really should be usefull outside of this project, too", we put it there. Generalization happens when we need it, that is, when reuse actually happens.

Then again, if the XP teams are refactoring mercilessly, aren't we digging the ground from component-based reuse?

No. Exactly *because* of merciless refactoring, components with high potential of reuse *are* reused early and often and therefore tend to stabilize rather fast.

As far as I know, successful reuse programs are pretty much based on strictly controlled interface evolution.

Merciless refactoring doesn't mean *uncontrolled* refactoring...

[Mark Herschberg]

Originally posted by Ilja Preuss:

Sorry, I can't imagine a process entailing closer collaboration than XP.

I can. Everyone sits down and descripts the project architecture in great detail, down to every class and method signature. The people writing code are nothing more then trained monkeys who fill in the blanks. Its compelte collaboration because everything is well defined in the docuement and everyone agress to follow it. In XP (or open source) you don't know what classes and method you need until you actually need them. Now this method is impractical in many cases for a number of reasons, but I believe it does provided closer collaboration then XP or many other methodologies.

Originally posted by Ilja Preuss:

What I meant to ask was: How would you want to anticipate unpredictable changes?

Did you really just ask that? Please tell me you are being facetious. Dilbert was asked the very same question by the pointy haired boss! Obviously you can't. Here's what I'm trying to sya by this...
Each process has a range over which its optimized. I'd be hesitant to use XP for software deployed in life-threatening mission ciritcal apps. XP doesn't work as well (cost-effectiveness) as other methods. Similarly I think if the project potentially faces major changes, I don't think XP is the best choice.

Originally posted by Ilja Preuss:

They pause the discussion when they feel they can estimate the story.
...
They stop the discussion when they refactored the code to the simplest form they can imagine.

This scares me. When can you estimate the story? If the team is too junior, they overestimate their abilities and don't spend enough time planning. Or they go off in different directions. Maybe it's just because the last few years I saw too many startups where the average age of a developer was 21. I fear XP would not work well in those situations. not enough discussion would be done.

Originally posted by Ilja Preuss:

Mhh, I wonder what might be a good alternative to giving people lots and lots of feedback on how they do...

A better signal to noise ratio. Lots of feedback is not necessarily optimal.
--Mark

[Junilu Lacar]

Originally posted by Mark Herschberg:
I can. Everyone sits down and descripts the project architecture in great detail, down to every class and method signature. The people writing code are nothing more then trained monkeys who fill in the blanks. Its compelte collaboration because everything is well defined in the docuement and everyone agress to follow it. In XP (or open source) you don't know what classes and method you need until you actually need them. Now this method is impractical in many cases for a number of reasons, but I believe it does provided closer collaboration then XP or many other methodologies.

I think I'm beginning to see why you are so skeptical about XP. What you describe seems a lot to me like Big Design Up-Front (BDUF).
Many people have found that this approach does not work very well. I have never seen it work well myself.
The problem with treating the people writing the code as being nothing more than trained monkeys is that they just simply aren't (although I think I may have run into one or two in disguise over the years ). Most "architects" are smart people but they are not omniscient--there will invariably be something that they miss up front that will come back and throw a monkey wrench in the works. And it's usually the developers who have to figure out how to get that wrench out. As a developer, I have often thought more of monkeys than of the so-called "architects" whose designs I had to implement. And then again, it isn't even the architect's fault sometimes. Sometimes a requirement is just not discovered until you are well into development. So how can you possibly describe "the project architecture in great detail, down to every class and method signature" ?
When I read XPE, it was like somebody had answered most, if not all, of my questions on how to make things better. It's just too bad I have only been able to experience it vicariously though.
Perhaps you have had much better experiences with the "heavy" methodologies. If so, it would be really interesting to hear more about what you and your team did or didn't do.

[Mark Herschberg]

Originally posted by Junilu Lacar:

I think I'm beginning to see why you are so skeptical about XP. What you describe seems a lot to me like Big Design Up-Front (BDUF).

I think you're misunderstanding me...
I claime dthat one issue in XP is that there is no central vision. In BDUF the vision is fairly well spelled out. In XP there is a "vision" in the sense that the final version exists, its just not visible yet. So people kinda move in the right direction. For experienced people, they tend to more to this final goal more directly then inexperienced people.
Not that in both cases people get "off the path" and in both cases inexperienced mor ehten expeirenced. I just think that people get further off in XP then in BDUF.
I'm not arguing for BDUF as better then XP. Also "big" is a relative term. I used the extreme version of BDUF to countering Ilja's argument that you can't get people to collaborate more closely (which I intepreted to mean, "agreeing on the final vision"). I would generally never advocate massive BDUF on any project, nor do I think of developers as code monkeys (read my posts in Job Discussion, I argue that developers need to get better business skills); but in theory, if you could do it, so that every thing is well specified from the start, then its extremely unlikely that people will "get off the path." Hence, you will have better collaboration.
Now I'm am arguing that given complex projects, or major changes during a project, in conjunction with a lack of capabilities (usually correlated to lack of experience), an XP team may not function as well as one which does more up front design (not massive, just sufficent) then XP would.
--Mark

[Ilja Preuss]

Originally posted by Mark Herschberg:
I claime dthat one issue in XP is that there is no central vision. In BDUF the vision is fairly well spelled out. In XP there is a "vision" in the sense that the final version exists, its just not visible yet.

To me, it looks quite the other way: in XP, the vision is "lived" by the team, reiterated and refined in daily discussions. There are Pair Programming discussions as well as diagrams on the white board from recent "quick design sessions". Misunderstandings and misconceptions are found out about early and are rather easy to fix.

Not that in both cases people get "off the path" and in both cases inexperienced mor ehten expeirenced. I just think that people get further off in XP then in BDUF.

Again, my experience is quite the opposite. A BDUF design contains much more speculation *and* is more costly to fix. "Hell, I think this class should really be split up." "Damn, we already spent two weeks on this design. Do you really want to update all the class, interaction etc. diagrams *again*, just to split this class?"

in theory, if you could do it, so that every thing is well specified from the start, then its extremely unlikely that people will "get off the path."

But that's a big "if", isn't it?

Hence, you will have better collaboration.

Well, your definition of "collaboration" must be quite different from mine...

Now I'm am arguing that given complex projects, or major changes during a project, in conjunction with a lack of capabilities (usually correlated to lack of experience), an XP team may not function as well as one which does more up front design (not massive, just sufficent) then XP would.

And I am arguing that XP *is* doing "just sufficient design"; though I concede that it might look scary to someone used to less agile development.

[Gavin Bong]

Again, my experience is quite the opposite. A BDUF design contains much more speculation *and* is more costly to fix. "Hell, I think this class should really be split up." "Damn, we already spent two weeks on this design. Do you really want to update all the class, interaction etc. diagrams *again*, just to split this class?

Updating the UML diagrams is actually a moot point. My modelling tool just sucks up the java class files and reverse engineer that to generate the diagrams.
Anyway, jumping right into the BDUF vs XP debate; I think there exists a middle ground b/w the two that works best. Architectural issues NEEDS to be designed up front. And one does not usually go writing everything from scratch. A pattern centric design process means that developers already have common viewpoints on how to code to meet requirements. The XP way would be to write the tests and get a skeleton class working; then refactor to patterns (that's not efficient enough, in my opinion)
Gavin

[Ilja Preuss]

Originally posted by Mark Herschberg:
Everyone sits down and descripts the project architecture in great detail, down to every class and method signature. The people writing code are nothing more then trained monkeys who fill in the blanks. Its compelte collaboration because everything is well defined in the docuement and everyone agress to follow it. In XP (or open source) you don't know what classes and method you need until you actually need them.

Yes, and in XP you "find" these classes/methods by collaboration (discussions with pair programming partner, with coworkers on the white board or in the planning meeting etc.) instead of looking at the design document. Seems to be closer collaboration to me...

quote:
--------------------------------------------------------------------------------
Originally posted by Ilja Preuss:
What I meant to ask was: How would you want to anticipate unpredictable changes?
--------------------------------------------------------------------------------
Did you really just ask that? Please tell me you are being facetious.

No, that is exactly what I meant to ask.

Dilbert was asked the very same question by the pointy haired boss! Obviously you can't.

Yes, I remember that cartoon - and I remember thinking "well, that's exactly what we do in XP" when reading it...
The way we do it is holding the code as flexible as we know to do.

I'd be hesitant to use XP for software deployed in life-threatening mission ciritcal apps.

I would certainly want to add more safety nets, too.

XP doesn't work as well (cost-effectiveness) as other methods.

Huh?

Similarly I think if the project potentially faces major changes, I don't think XP is the best choice.

Again, we significantly disagree...

This scares me. When can you estimate the story?

When you can say wether a story is about as costly as other "one point stories", twice as costly or thrice as costly.

If the team is too junior, they overestimate their abilities and don't spend enough time planning.

There are three forces at work correcting their errors, here:
- they will plan often. Every week they will know how "correct" the plan was and have the option to replan. "JDBC is easier than I thought. I think we should re-estimate our database-stories."
- they will use abstract measures to estimate and gauge mapping to real time. "We managed to do 15 points last week. Next week, Daniel will be on vacation, so lets plan to do 13 points that iteration."
- they will still make errors in estimation. As they estimate rather small chunks of work, those errors will tend to balance each other.

Or they go off in different directions. Maybe it's just because the last few years I saw too many startups where the average age of a developer was 21. I fear XP would not work well in those situations. not enough discussion would be done.

That's what baffles me - you somehow seem to equate "writing design documents you then only need to code to" more to "discussion", than doing weekly planning meetings, daily stand-up meetings, pair programming, short design sessions on the white board etc.
I guess I just don't get it...

[Ilja Preuss]

Originally posted by Gavin Bong:
Updating the UML diagrams is actually a moot point. My modelling tool just sucks up the java class files and reverse engineer that to generate the diagrams.

Yeah, but of what use are those diagrams? For me, diagrams need to present an *abstraction* from the code to be usefull - if they map *directly* to the code, I could just look at the code anyway. I haven't yet found a tool being able to automatically preserve such an abstract point of view. (Or even *create* one from code.)
And what about those collaboration-, state-, etc. diagrams?

Anyway, jumping right into the BDUF vs XP debate; I think there exists a middle ground b/w the two that works best.

Of course - as long as you meant to say "BDUF vs. Design By Coding". XP isn't "jump directly to coding", after all.
I know of many developers, though, who after trying out the XP practices found that the middle ground could be much more on the side of "design while coding" than they previously thought.

Architectural issues NEEDS to be designed up front.

Example of such an architectural issue, please?

And one does not usually go writing everything from scratch. A pattern centric design process means that developers already have common viewpoints on how to code to meet requirements. The XP way would be to write the tests and get a skeleton class working; then refactor to patterns (that's not efficient enough, in my opinion)

I don't know where you got this from. You are right that you will often refactor to patterns. But if I "see" from the beginning that a certain pattern would be helpfull, I don't hesitate to let my tests "force" me to implement it. I still lookout for smells in my code telling me that a different pattern could be better, though.
It's also quite common in planning/design discussions to hear phrases like "That's easy - we will just give Gnoom a FoobulatorFactory, that's it."

[Ilja Preuss]

Somehow missed this in my previous reply (reply with code isn't working very well... )...

Originally posted by Mark Herschberg:
A better signal to noise ratio. Lots of feedback is not necessarily optimal.

Of course I was speaking about "lots of usefull feedback"...

[Mark Herschberg]

Originally posted by Ilja Preuss:

To me, it looks quite the other way: in XP, the vision is "lived" by the team, reiterated and refined in daily discussions. There are Pair Programming discussions as well as diagrams on the white board from recent "quick design sessions". Misunderstandings and misconceptions are found out about early and are rather easy to fix.

That seems to be the crux of the disagreement. I claim inexperineced teams will not do as well under XP.

Originally posted by Ilja Preuss:
But that's a big "if", isn't it?

Yes, which is why I would never advocate big upfront development. I used that example for argumentative purposes only (see my posting the Stealth XP thread, I made similar arguments there).

Originally posted by Ilja Preuss:
And I am arguing that XP *is* doing "just sufficient design"; though I concede that it might look scary to someone used to less agile development.

You see design which was created and cast aside as wasted. I see it as experienced learned which will help the team do better. Much like science, a wrong path may still be valuable. The trivck is finding the balance and quickly recognizing that the path is wrong.

Originally posted by Gavin Bong:

Anyway, jumping right into the BDUF vs XP debate; I think there exists a middle ground b/w the two that works best.

Yes, I agree! My point is for a less experienced team, more upfront design is needed.

Originally posted by Ilja Preuss:
The way we do it is holding the code as flexible as we know to do.

There is nothing inherent in upfront design which makes the code inflexibile. Now if you have a lot for formal documents which need to be updated everytime you make a change, then that's not very flexibile. Most of us who practice upfront design know how to avoid this.

Originally posted by Ilja Preuss:

I'd be hesitant to use XP for software deployed in life-threatening mission ciritcal apps.
I would certainly want to add more safety nets, too.
XP doesn't work as well (cost-effectiveness) as other methods.
Huh?

By cost effective I mean that the cost for a bug in such a project is severe (lawsuits, plane crash, etc). I don't think companies trust XP enough to use it on such projects. When comapnies do successfully use it on mission critical projects, only then will I believe it works in these circumstances.

Originally posted by Ilja Preuss:
There are three forces at work correcting their errors, here:

And all those three forces can exist in a process which also used upfront design. I don't see any advantage of XP here.

Originally posted by Ilja Preuss:
That's what baffles me - you somehow seem to equate "writing design documents you then only need to code to" more to "discussion", than doing weekly planning meetings, daily stand-up meetings, pair programming, short design sessions on the white board etc.

As I noted in the other thread, the extreme example of document until everything is totally understood is used for argumentative purposes only. I have never advocated that as a practical method. You can (and if you want any hope of succeeding) should have regular meetings, reviews, design sessions etc in any process. I just happen to think doing some upfront design in beneficial to XP for certain project/team combinations. Likewise, for a significantly big change halfway through the project, such significant design needs to be done again.

Originally posted by Ilja Preuss:

My experience with people and skillsets is that their shortcomings won't be overcome by the XP model under the circumstances I described earlier.
Mhh, I wonder what might be a good alternative to giving people lots and lots of feedback on how they do...

Oh, you meant really good feedback! And I meant really good design, one with no bugs. :-p Again, I believe that the team I described would not be very efficent at generating such feedback.
--Mark

[Junilu Lacar]

Originally posted by Mark Herschberg:

Oh, you meant really good feedback! And I meant really good design, one with no bugs. :-p Again, I believe that the team I described would not be very efficent at generating such feedback.
--Mark

Likewise, the team that you described would not be efficient at generating a good design either!
Without proper guidance, an inexperience team will not be efficient, no matter what process they follow. The process that ultimately allows them to learn faster will be more efficient. To me all the arguments you had against XP actually make the case for it with inexperienced teams.
Regarding wasted effort: no, (and I probably speak for Ilja too) I don't think that designs that are thrown out are wasted. They are, as you said, a necessary part of the learning process. You just have to be efficient and control how much of the design you throw out. Unit tests up front would help inexperienced programmers to see their mistakes a lot quicker and more efficiently find flaws in their designs. True, inexperienced programmers may need more time to come up with a good design but that doesn't necessarily mean that they have to do more up front design.
There's the study that you want to do! Whether an inexperienced team truely will not do as well with XP than with more "traditional" methods.

[Mark Herschberg]

Originally posted by Junilu Lacar:

Without proper guidance, an inexperience team will not be efficient, no matter what process they follow. The process that ultimately allows them to learn faster will be more efficient.

Right, and my whole claim was simpyl that for an inexperienced team (which so many teams seem to be these days) some up-front design is better then pure XP. (I acually calimed it when there was a major design change, but I think the principle is the same.)
--Mark

[Lasse Koskela]

Originally posted by Junilu Lacar:
...I don't think that designs that are thrown out are wasted. They are, as you said, a necessary part of the learning process. You just have to be efficient and control how much of the design you throw out.

What do you mean by "controlling" here? I trust that you don't mean having some corporate rules for "throwing out a maximum of 26% of design"...
I believe we all agree that what you throw away should be dictated by the compromise between "perfectionism" and "reality" (full rewrite based on latest knowledge versus maximum reuse in seeking short-term financial benefits).

[Ilja Preuss]

Originally posted by Lasse Koskela:
What do you mean by "controlling" here? I trust that you don't mean having some corporate rules for "throwing out a maximum of 26% of design"...

I guess what Junilu meant was *reflecting* about it.
That is, if you throw away "upfront designs", you should ask yourself: How much did it help me nevertheless? Was it worth the effort? What alternatives exist?
Of course the same is true for "big refactorings" - if you find out you need to do bigger changes to an existing design, you should ask yourself: Where is the big change coming from? Was the old design somehow "wrong"? What could have helped me to reduce the costs of the change?

[Ilja Preuss]

Originally posted by Mark Herschberg:
You see design which was created and cast aside as wasted.

Not inherently wasted, though I would certainly like to reflect on its value.
In fact, you could probably call the designs I start *coding* to be "cast aside" to be refactored to better ones.

I see it as experienced learned which will help the team do better. Much like science, a wrong path may still be valuable. The trick is finding the balance and quickly recognizing that the path is wrong.

Yes. Part of that trick is finding the best medium to express the design. And to me the optimal medium often seems to be actual code.

--------------------------------------------------------------------------------
Originally posted by Ilja Preuss:
The way we do it is holding the code as flexible as we know to do.
--------------------------------------------------------------------------------
There is nothing inherent in upfront design which makes the code inflexibile.

So, do you withdraw your statement "Obviously you can't [anticipate unpredictable changes]"?
BTW, I think Big Up Front Design makes flexibility less likely: Wether the code is flexible is best seen by looking at the actual code, so you need the feedback from the code to make it flexible. The more you are delaying that feedback, the less likely it is to be accounted for (if only for psychological reasons).

quote:
--------------------------------------------------------------------------------
Originally posted by Ilja Preuss:
There are three forces at work correcting their errors, here:
--------------------------------------------------------------------------------
And all those three forces can exist in a process which also used upfront design. I don't see any advantage of XP here.

The question wasn't about upfront design here, but about wether the XP planning process scares you. Does it still scare you?

You can (and if you want any hope of succeeding) should have regular meetings, reviews, design sessions etc in any process. I just happen to think doing some upfront design in beneficial to XP for certain project/team combinations.

As I already stated in another thread, design discussions etc. *are* part of XP. This is gets probably most obvious in the System Methaphor practice.

Likewise, for a significantly big change halfway through the project, such significant design needs to be done again.

Last week, we just did a 1,5 hour design discussion to think about harmonizing a certain aspect of three til know separated projects. Are you thinking of something similar?

Oh, you meant really good feedback! And I meant really good design, one with no bugs. :-p Again, I believe that the team I described would not be very efficent at generating such feedback.

So you think that it would be more efficient at generating a really good design, one with no bugs and *afterwards* coding it? What makes you think so?