Software process - recommended books?

[Tim West]

Hi guys,

Through my work I have access to an online library of IT texts. I've just read Fowler's "Refactoring" and Kent Beck's "Extreme Programming Explained".

Does anyone have a recommendation for another good book to read? I'm basically interested, at the moment, on more high-level process stuff, or anything a little out of the ordinary.

Although the library's good, it let me down on The Mythical Man Month, or I'd read that...I'll have to buy it I guess.

Any quick recommendations would be much appreciated.


--Tim

[Lasse Koskela]

Here's some books, from the top of my head, that I would recommend:

"Agile and Iterative Development" (Larman)
"Agile Software Development" (one by Alistair Cockburn, another by Robert C. Martin)
"Lean Software Development" (Poppendieck)
"Balancing Agility and Discipline" (Boehm, Turner)
"Peopleware" (Demarco, Lister)
"Slack" (Demarco)
"Agile Management" (Anderson)
"Software by Numbers" (Denne, Cleland-Huang)
"Lean Thinking" (Womack)

[Ilja Preuss]

"Artful Making" is interesting, too. Also heard good things about "Software Craftsmanship".

[Tim West]

Thanks for the advice guys. I'll have a look at those.


--Tim

[Jeff Langr]

I'll suggest The Pragmatic Programmer by Dave Thomas and Andy Hunt as an interesting read.

-j-

[Richard Leavitt]

Lasse's list is a "best of." I also liked
Highsmith: Agile Project Management, and
Schwaber: Agile Project Management with Scrum

[Lasse Koskela]

Hmm. On a second thought, "Lean Thinking" might be a bit too "boring" for some people (it's not about software development) and the ideas should come through from Poppendiecks' book.

[Tim West]

Thanks again everyone

Having been briefly distracted by Bloch's 'Effective Java', I've now got bookmarks in Larman's 'Agile & Iterative Development', McConnell's 'Professional Software Development', Hunt & Thomas's 'The Pragmatic Programmer and Denne & Cleland-Huang's 'Software By Numbers.

Sure is a lot to learn eh?



--Tim
[ June 29, 2004: Message edited by: Tim West ]

[Ilja Preuss]

Originally posted by Tim West:
Sure is a lot to learn eh?

And it never stops!

[frank davis]

Originally posted by Tim West:
Hi guys,

Through my work I have access to an online library of IT texts. I've just read Fowler's "Refactoring" and Kent Beck's "Extreme Programming Explained".

Does anyone have a recommendation for another good book to read? I'm basically interested, at the moment, on more high-level process stuff, or anything a little out of the ordinary.

Although the library's good, it let me down on The Mythical Man Month, or I'd read that...I'll have to buy it I guess.

Any quick recommendations would be much appreciated.


--Tim

Mythical Man Month is way overrated. The main take home point from that book is simply Brook's Law. If you want an overview of software development, see "Rapid Development" by Steve McConnel. Not a short read, but that's to be expected since it combines comprehensiveness with details and references to specific studies that make the book completely unparalelled. Steve also writes in a readable style.

[Warren Dew]

herb slocomb:

Mythical Man Month is way overrated. The main take home point from that book is simply Brook's Law.

Gee, I got a lot more out of the book than that. I had to look up Brook's law, where I remember offhand the issue of the "second system" effect, issues surrounding unity of design, and in the current edition, the "no silver bullet" argument - the last of these probably being the most important to understand.

I do agree that many of Brooks' ideas are currently unpopular, and certainly to some extent in opposition to ideas from the 'agile development' movement, but that doesn't make them wrong.

[Lasse Koskela]

Originally posted by Warren Dew:
I do agree that many of Brooks' ideas are currently unpopular, and certainly to some extent in opposition to ideas from the 'agile development' movement, but that doesn't make them wrong.

There's no silver bullet

[Junilu Lacar]

Originally posted by Warren Dew:
I do agree that many of Brooks' ideas are currently unpopular, and certainly to some extent in opposition to ideas from the 'agile development' movement, but that doesn't make them wrong.

Hmmm, I remember highlighting a lot of things in the book that seemed to foreshadow many of the XP/agile ideas (IIRC, mostly from the "No Silver Bullet" paper).

Which of Brook's ideas are you referring to?

[Warren Dew]

Junilu Lacar:

Which of Brook's ideas are you referring to?

The primary one I remember is the argument that one will get a better piece of software with stronger conceptual integrity if one or a small number of architects to complete the design before implementation, in his chapter "Aristocracy, Democracy, and System Design".

In general, he emphasizes documentation a lot more than I think most Agile practitioners do.

His "surgical team" suggestion takes almost the opposite approach from XP - instead of everyone coding together, only one person writes the application code with everyone else supporting him.

A lot of the book seems to assume a "waterfall" approach.

Note even if these ideas are in opposition to the "agile development" philosophy, that doesn't mean one of them has to be wrong. It seems to me that most of Brook's suggestions make sense for making beautiful, high quality software to satisfy relatively slowly changing requirements; agile development seems to me to be more geared towards making satisfactory software in response to rapidly changing requirements.

[Warren Dew]

And for a book on a software development technique that seems to be coming back into vogue in some quarters, try Death March by Edward Yourdon.

[Ilja Preuss]

Originally posted by Warren Dew:
Note even if these ideas are in opposition to the "agile development" philosophy, that doesn't mean one of them has to be wrong. It seems to me that most of Brook's suggestions make sense for making beautiful, high quality software to satisfy relatively slowly changing requirements; agile development seems to me to be more geared towards making satisfactory software in response to rapidly changing requirements.

Agile development also pays back when requirements are "written in stone", but our understanding of them, or our understanding of how to best implement them is not. That is, Agile development allows the whole team to learn during the development *and* let the new learning to be accounted for.

Important to understand is, though, that this only works with rapid feedback and cheap iterations. These things certainly changed quite dramatically since the book was written.

[Warren Dew]

Ilja Preuss:

Agile development also pays back when requirements are "written in stone", but our understanding of them, or our understanding of how to best implement them is not. That is, Agile development allows the whole team to learn during the development *and* let the new learning to be accounted for.

I think this situation puts the difference between the Agile approach and the Brooks' ideas into high relief. The Agile approach - or at least the more extreme versions - would have you bring on all the engineers at the beginning; they get to learn the requirements together, and make a bunch of mistakes together. Brooks' approach would have the learning done by a small group that specializes in understanding requirements; that saves a lot of effort in that only that small group has to make the mistakes, and once they've figured out the requirements, can turn them over to the complete implementation group without the implementation group having to experience all the dead ends first hand.

Put another way, while the waterfall approach may be more efficient when the requirements are unchanging, the agile approach can also handle that situation. In contrast, when the requirements do change rapidly - far more common today - the agile approach is not only more efficient, but it's questionable whether the waterfall approach will work at all.

[Lasse Koskela]

Originally posted by Warren Dew:
Brooks' approach would have the learning done by a small group that specializes in understanding requirements; that saves a lot of effort in that only that small group has to make the mistakes, and once they've figured out the requirements, can turn them over to the complete implementation group without the implementation group having to experience all the dead ends first hand.

How did Brooks think that this small group would be able to transfer the knowledge they had accumulated to the rest of the implementation group without making those mistakes? As we all very well know, knowledge is not something you can just pour into someone's head...

Originally posted by Warren Dew:
Put another way, while the waterfall approach may be more efficient when the requirements are unchanging, the agile approach can also handle that situation. In contrast, when the requirements do change rapidly - far more common today - the agile approach is not only more efficient, but it's questionable whether the waterfall approach will work at all.

Good point.

[Ilja Preuss]

Originally posted by Warren Dew:
Brooks' approach would have the learning done by a small group that specializes in understanding requirements; that saves a lot of effort in that only that small group has to make the mistakes, and once they've figured out the requirements, can turn them over to the complete implementation group without the implementation group having to experience all the dead ends first hand.

This not also assumes, as Lasses already pointed out, that the requirements can be flawlessly communicated from the requirements gatherers to the implementors. It also assumes that you *can* fully understand the requirements without trying to implement them. I don't think this is often the case; in fact experience seems to suggest that one of the most effective ways to strengthen your understanding of the requirements is presenting the users an early working system.

[Warren Dew]

Lasse Koskela:

How did Brooks think that this small group would be able to transfer the knowledge they had accumulated to the rest of the implementation group without making those mistakes?

Brooks advocates written specifications. Irrespective of the specific method, the waterfall approach is more efficient than the agile approach - in this now unusual case of unchanging requirements - as long as this transfer method is more efficient than the initial exploration. That seems likely, since the architects will be able to warn the implementors of pitfalls and thus help them avoid wasting as much time. (Note that I'm using "architect" in Brooks' sense of "specify the user interface", not to mean "design the code".)

Ilja Preuss:

It also assumes that you *can* fully understand the requirements without trying to implement them. I don't think this is often the case; in fact experience seems to suggest that one of the most effective ways to strengthen your understanding of the requirements is presenting the users an early working system.

Brooks' experience obviously differs.

My own experience is that, in cases where the requirements really are "written in stone" - which is the situation we're discussing - the architect first and develop afterwards approach really is more efficient. The problem is that this case, in my experience, is quite rare. Far more common is the situation where customers think their requirements are "written in stone" but are wrong, which is probably the situation where the waterfall approach tends to fail most spectacularly.

[Ilja Preuss]

Originally posted by Warren Dew:
Brooks' experience obviously differs.

It certainly does!

But perhaps to some amount it differs because things have changed since the book was written. Today, computers are orders of magnitudes faster *and* cheaper. Additionally, we have advanced in using OO and other techniques to decouple our designs.

That is, iterations can be much faster and cheaper today. The faster and cheaper iterations are, the more you can affored to experiment, to work feedback-driven instead of plan-driven.

[Lasse Koskela]

I guess you've both heard this a dozen times before, but Royce (the author of the so-called "waterfall paper" for DoD) was saying that 1) waterfall is good for only the simplest of projects, and 2) you still have to "do it twice".

[Warren Dew]

I'm using "waterfall" in the more general sense of an overall noniterative process where the architecture - the detailed user interface design - precedes implementation - where implementation includes code design, writing, and testing. I'm open to suggestions for other terms to use if that's confusing. (Noniterative? Sequential?) I agree that Royce's nine step process seems to be overly constrained, even with the iterative elements that it included.

Beck says XP doesn't work for very large projects, either. Maybe very large projects are just doomed to failure.
[ July 04, 2004: Message edited by: Warren Dew ]

[Lasse Koskela]

Originally posted by Warren Dew:
Beck says XP doesn't work for very large projects, either. Maybe very large projects are just doomed to failure.

Yep. A team using any process will have a hard time delivering a large project. That's why we call them "large". I believe the key to successful large projects is "divide and conquer"...

[Ilja Preuss]

Originally posted by Warren Dew:
Beck says XP doesn't work for very large projects, either.

That's what Beck wrote years ago. Since then XP has been used on bigger and bigger projects. As far as I know there are several successful projects at the size of around 50 developers which used an XP-like process.

On the other hand it depends on what you mean by "very large". I know of a coach currently writing a book about her experience of "Agile Software Development in the Large", with teams of a size of (IIRC) 150 developers. When talking to another coach she had to hear "150 developers are peanuts - several thousand developers make a large project"...

Maybe very large projects are just doomed to failure.

Well, there is the saying that every successful large project started as a small project which grew over time...

[Lasse Koskela]

I know of a coach currently writing a book about her experience of "Agile Software Development in the Large", with teams of a size of (IIRC) 150 developers.

I assume you're talking about Jutta? Her book is out already. For some reason it's not listed properly at Amazon.com but it is out (I've seen it).

[Ilja Preuss]

Originally posted by Lasse Koskela:

I assume you're talking about Jutta? Her book is out already. For some reason it's not listed properly at Amazon.com but it is out (I've seen it).

Yes, that's what I was talking about! Thanks for the reminder...

[frank davis]

Originally posted by Warren Dew:
herb slocomb:

Mythical Man Month is way overrated. The main take home point from that book is simply Brook's Law.

Gee, I got a lot more out of the book than that...

I do agree that many of Brooks' ideas are currently unpopular, and certainly to some extent in opposition to ideas from the 'agile development' movement, but that doesn't make them wrong.

I didn't say Brook was wrong; its just that the "average" developer today is not going to get much use from his ideas and experiences. Let's look at his ideas :

1. "Surgical teams" - 10 people on a team, yet only 1 person responsible for the code. Yes, we know certain programmers are extremely productive, but having one chief programmer do nearly all the designing and coding will not work consistently - there simply aren't enough of those talented programmers around in the industry.

2. "Conceptual Integrity" - Brooks was referring to the system architecture and by that he meant the specification of the user interface. Unlike 1975 however, in 2004 there now exists a large number of well understood user interface conventions. Many of MS Windows's UI conventions are used in other OS's ("Lindows"), etc. We have web browser's with conventions for back buttons, reload, etc, etc. Things we take for granted as far as UI design simply didn't exist in 1975 and there's less danger now of some geek coming up with a completely new and unconventional UI that would perplex the user (I know everyone has a few contrary examples...)

3. "Second System Effect" - whereby the next software version gets bloated with too many dubious 'features'. OK, this is a valid point, but its just that bloatware seemed such an obvious thing today that it seemed obvious to me. I guess since he's writing in 1975, there's little to say that will seem new to those of us with a little experience in 2004.

4. Documentation - Brook's big on documentation, even going so far to advocate telephone logs. I don't necessary disagree with him, but just don't aggree to the same degree. Remember that's Brook's experiences are from huge IBM OS projects, such as the OS/360 mainframe project with 1000 people on a team. Most developer's don't work on 1000 member teams and his emphasis on many aspects of development is misplaced when considering much smaller teams.

5. "Plan to Throw One Away" - Brook's book was never revised unfortunately, yet Brook himself recanted this concept in a "Where I went Wrong" section of a document presented to the IEEE in 1991.


Brooks writes from a distant time and experiences where punch cards on mainframes ruled the world, with huge development teams, where programmer productivity was a fraction of what it is today, when writing in assembly was not uncommon, where debugging tools uncommon, etc. I enjoyed the book and the history, he's a good writer, and maybve it is needed for perspective at least, but in day to day life there is little of value.

[Scott Ambler]

The following sites may prove to be of interest:
www.agilemodeling.com (effective modeling and documentation processes)
www.agiledata.org (effective data-oriented processes)
www.enterpriseunifiedprocess.com (extension to the RUP for real-world orgs)

Hope they help.

- Scott