Why software estimates are often wrong

[fred rosenberger]

Here is an interesting analogy.

It got me thinking that software projects are sort of fractal like...as you zoom in to see more details of a given sub-section, you see more twists and turns, but the general look is still the same.

[Seetharaman Venkatasamy]

most of the time , people pressure is more than technical pressure! no? ;)

[Pat Farrell]

The estimates are junk because there is no engineering in software engineering. If you want to build a building, you can tell the architect what your budget is, and he will design a building to that amount. This is because the building trades have thousands of years experience in estimating and pricing.

No one who asks to have a custom house built will expect the builder to come back when its half way finished asking "do you want a kitchen in your house" but we do it all the time with software.

[Maneesh Godbole]

Since this is a public forum, I got perfect systems in place where the management defines perfect, detailed and final requirements on time. Then we thrash it out perfectly and come up with a perfect framework. Then we code it perfectly. All this perfection leads to a winning combination! I am proud to be a part of such an awesome team!

[Pat Farrell]

@Maneesh, are you dead and in Heaven?

[Maneesh Godbole]

@Pat,
Nope. I am posting on a public forum

[Frank Silbermann]

Pat Farrell wrote:The estimates are junk because there is no engineering in software engineering. If you want to build a building, you can tell the architect what your budget is, and he will design a building to that amount. This is because the building trades have thousands of years experience in estimating and pricing.

No one who asks to have a custom house built will expect the builder to come back when its half way finished asking "do you want a kitchen in your house" but we do it all the time with software.

If you ask a game developer to build you a game that has a list of features selected from various games he has already developed, he can give you a good estimate.

Suppose, on the other hand, you tell an architect that you want him to design for you a factory for you to produce a new kind of widget, and

(1) You can tell him _some_ of the operations that go into producing widgets -- but you cannot think of all of the steps right now and will add new operations as you think of them, and

(2) Some of the operations you describe in the wrong order (but you will correct that as the design nears completion).

Will the architect be able to accurately predict the time and cost of giving you the design?

What if the firm offers both architectural and construction services, and some of the specifications will not be nailed down until well into the construction phase? Can he give you a good upfront estimate of the total time and cost?

What if you hire your own architects and construction workers to design an build a type of building with features they've never before designed or built, perhaps using a new construction material that has just been invented so they've never used it before. Will they be able to give you a good upfront estimate of the total time and cost required -- assuming they use a proper "engineering discipline"?

[Pat Farrell]

Frank Silbermann wrote:you tell an architect that you want him to design for you a factory for you to produce a new kind of widget,
.....
Will the architect be able to accurately predict the time and cost of giving you the design?

Yes, absolutely. But the architect designes the building, not the process machines.

The software industry constantly misuses the term "architect" and the roles that an architect has. In the building word, its well defined.

[Frank Silbermann]

Pat Farrell wrote:

Frank Silbermann wrote:you tell an architect that you want him to design for you a factory for you to produce a new kind of widget,
.....
Will the architect be able to accurately predict the time and cost of giving you the design?

Yes, absolutely. But the architect designes the building, not the process machines.

The software industry constantly misuses the term "architect" and the roles that an architect has. In the building word, its well defined.

I suppose, if the changes in process do not dictate differences in the layout of the building (electricity connections, water connections, size). Now what about the industrial engineer who designs the floor layout?

[Pat Farrell]

Frank Silbermann wrote: I suppose, if the changes in process do not dictate differences in the layout of the building (electricity connections, water connections, size). Now what about the industrial engineer who designs the floor layout?

Have you ever been in a factory? They are open and can run reasonable amounts of power, water, waste, etc. anywhere they want. Typically they have big distribution panels at strategic locations. If they need to, whole sections can be demolished and rebuilt.

Designing a process for the factory is a challenge, and I'm sure they run over budget all the time. Take the plant where they first made Tyvek. No one had every done it before, except in a test tube at small scale. That had to have been a multi-year effort.

My point is that unlike housing, where we humans have spent thousands of years talking about what we want in a house, we have lack the communication skills to talk about what we want software to do.

Worse, the economics are all wrong. The pressure is to get the software out the door as quickly and cheaply as possible. There is no economic pressure to build with concern for it being robust, scalable, modular, extensible, etc.

[Frank Silbermann]

Pat Farrell wrote:

Frank Silbermann wrote: Now what about the industrial engineer who designs the floor layout?

Designing a process for the factory is a challenge, and I'm sure they run over budget all the time. Take the plant where they first made Tyvek. No one had every done it before, except in a test tube at small scale. That had to have been a multi-year effort.

My point is that unlike housing, where we humans have spent thousands of years talking about what we want in a house, we have lack the communication skills to talk about what we want software to do.

Worse, the economics are all wrong. The pressure is to get the software out the door as quickly and cheaply as possible. There is no economic pressure to build with concern for it being robust, scalable, modular, extensible, etc.

I don't think communication skills are the issue. Any communication modality that is sufficiently detailed to specify a program becomes, potentially, a programming language in its own right. If the design of new factory processes for new products run over budget all the time, we don't attribute _that_ to inadequate communication skills, do we?

Your point about the lack of economic pressure to build with a concern from stability, scalability, extensibility, flexibility, etc., is a valid managerial issue. I suspect that in most engineering disciplines, many eyes evaluate and refine a low-level individual contributor's engineering design in detail before it is thrown over the wall to manufacturing. A precondition for the economic incentives you mention is for all code to be read and understood at the line level by those with the power to hire, fire, reward and punish. (Otherwise, investing in software is as risky as hiring an adviser to invest your money in vehicles about which you are completely ignorant. A sizable percentage of people who do so end up losing their money.)

Adopting such a practice for software might triple the cost. By skipping those steps, managers are basically accepting significant risk in the hope of saving 67% of the cost. Sometimes they win and sometimes they lose.

[Pat Farrell]

Frank Silbermann wrote:I don't think communication skills are the issue. Any communication modality that is sufficiently detailed to specify a program becomes, potentially, a programming language in its own right.

Yes, and that is not the case with "build me a house"
And when building a house, we have the "you want a kitchen" issue.

That I call communications problems. You are free to label it anything you want, but until we solve it, and can communicate what we want, and can measure it, all software estimates will keep being wrong.

When you can measure what you are speaking about, and express it in numbers, you know something about it; but when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind. It may be the beginning of knowledge, but you have scarcely, in your thoughts, advanced to the stage of science.

Popular Lectures and Addresses, Lord (William Thomson) Kelvin.

[Frank Silbermann]

Pat Farrell wrote:

Frank Silbermann wrote:I don't think communication skills are the issue. Any communication modality that is sufficiently detailed to specify a program becomes, potentially, a programming language in its own right.

Yes, and that is not the case with "build me a house"
And when building a house, we have the "you want a kitchen" issue.

That I call communications problems. You are free to label it anything you want, but until we solve it, and can communicate what we want, and can measure it, all software estimates will keep being wrong.

I don't think that's because we communicate better with houses; I think it's because the customer and provider start off with rich shared assumptions as to what constitutes a "house." That is in part because the overwhelming majority of us have used houses at some point in our lives. When you ask a developer to write, say, a sales management program, chances are the developer has never used a sales management program (and often neither has the customer).