This week's book giveaway is in the OCPJP forum.
We're giving away four copies of OCA/OCP Java SE 7 Programmer I & II Study Guide and have Kathy Sierra & Bert Bates on-line!
See this thread for details.
The moose likes Agile and Other Processes and the fly likes Iterative or waterfall ? Big Moose Saloon
  Search | Java FAQ | Recent Topics | Flagged Topics | Hot Topics | Zero Replies
Register / Login


Win a copy of OCA/OCP Java SE 7 Programmer I & II Study Guide this week in the OCPJP forum!
JavaRanch » Java Forums » Engineering » Agile and Other Processes
Bookmark "Iterative or waterfall ?" Watch "Iterative or waterfall ?" New topic
Author

Iterative or waterfall ?

Kodo Tan
Ranch Hand

Joined: Aug 14, 2001
Posts: 105
Hi all
I have been pondering this for quite some time and maybe you guys can help me on this:
(1) We all know that for MOST modern project management, we should avoid waterfall methodology as it is restrictive, not adaptable to use changes, and other classic bad points about waterfall model easily found in any software project management books.
(2) For OO/J2EE projects, we are advised that one good approach is OOAD or iterative model and blah blah blah.
(3) Now here is my concern: In most projects teams I'm in, if I examine the schedule and resources planning carefully, I think it's more towards waterfall approach than anything else. For instance, for most project schedules, it is quite common to find the following:
(a) Project Initialisation Jan - Feb
(b) Users requirements Feb - Mar
(c) Functional specifications Mar - Apr
(d) Detailed Design Spec Apr - Jun
(e) Code Development Jun - Oct
(f) SIT Oct - Nov
(g) UAT Nov - Dec
(h) Prod Dec

Aren't the above a waterfall approach? If it's an iterative model, then where is
the schedule for iteration? :-)
Isn't it strange if the PLs of the teams use
such "waterfall" approach to schedule and plan resources their stands are supposed to be iterative ? Are they confused :-) or they have
some good project management reasons (that I'm unaware) to do so ?
Mark Herschberg
Sheriff

Joined: Dec 04, 2000
Posts: 6037
You make a good point. Often times the tools are either designed for a waterfall process, or people are simply used to using them in those manners (because the people who taught them the tools used them that way). If you encounter such a schedule, I highly recommend that you point this out and try to have them schedule a more iterative approach, since that's what you'll likely do anyway (whether or not you have the schedule reflect it).

--Mark
Gerry Giese
Ranch Hand

Joined: Aug 02, 2001
Posts: 247
Seems like you're stuck somewhere in the middle - let's call it the 'whirlpool' model. Trying to meld waterfall and iterative can be difficult. RUP seems to have a fairly good approach to scheduling an iterative approach, and I've seen a few charts illustrating it that make it easier for people to understand how it works.
Question: are you able to choose the methodology, schedule, milestones, etc., or are you locked into some specific requirements by your employer(s)? Your project phase list is defintely waterfall. If you're stuck with it and micromanagement is aware of where you are and wants you to stick to the phases and schedule, then there's nothing you can do. If management stays out of it and just wants to see progess and have you meet your deadline, you can ignore the waterfall schedule and do the iterative thing. You can then show progress via your milestone releases.
Iterative can be used for OO and non-OO equally well, IMO. The toughest thing on any project is your (a) and (b). You can't really come up with a reasonable schedule until done those two the first time.
You can give a really rough estimate at the start as long as everyone agrees not to hold you to it. After (a) and (b) you should be able to break out the work chunks, pick out your iterative milestones for each chunk, and attach some measure of effort/resources required and calculate some dates. If you're lucky, you can talk your employers into making these 'tentative'.
By the time you've got into (c) and started to iterate back to (b), your dates should firm up even further. Something to remember is planning not just code iteration time, but for (b), (c), and (d). By the time you reach your first milestone you'll have enough perspective to know if you'll meet your tentative dates, but, as with just about any project, it may be too late to change expectations if dates are off by more than 15%.
Good luck, and let us know what happens!


CJP (Certifiable Java Programmer), AMSE (Anti-Microsoft Software Engineer)
Author of Posts in the Saloon
Shubhrajit Chatterjee
Ranch Hand

Joined: Aug 23, 2001
Posts: 356
I beg to differ...
I understand that most project managers plan their project in such a way that it looks like they are following a waterfall model... but they may be not !!!
Why I feel so...
1. Requirements definitely should not iterate.
2. Analysis also in my opinion should not iterate. Anyway it should not iterate for 90% of projects as it is directly derivable from requirements.
3. Design can iterate. However a competent SW firm will often freeze the design based on past experience and it is highly unlikely that it would change and thus iteration will not be necessary.
4. Construction will definitely iterate. This is a design principle we almost always follow. This is called a build cycle in the construction phase for iterative processes, and there will be multiple build cycles in the construction phase. In other words a construction phase can be said to constitute of multiple sub phases called build cycles...
5. Integration and System testing should happen only after all iterations ie all build cycles are over.
Thus I see no error in the project schedule given below... though it would have helped to divide the construction into build cycles.

Originally posted by Kodo Tan:
Hi all
I have been pondering this for quite some time and maybe you guys can help me on this:
(1) We all know that for MOST modern project management, we should avoid waterfall methodology as it is restrictive, not adaptable to use changes, and other classic bad points about waterfall model easily found in any software project management books.
(2) For OO/J2EE projects, we are advised that one good approach is OOAD or iterative model and blah blah blah.
(3) Now here is my concern: In most projects teams I'm in, if I examine the schedule and resources planning carefully, I think it's more towards waterfall approach than anything else. For instance, for most project schedules, it is quite common to find the following:
(a) Project Initialisation Jan - Feb
(b) Users requirements Feb - Mar
(c) Functional specifications Mar - Apr
(d) Detailed Design Spec Apr - Jun
(e) Code Development Jun - Oct
(f) SIT Oct - Nov
(g) UAT Nov - Dec
(h) Prod Dec

Aren't the above a waterfall approach? If it's an iterative model, then where is
the schedule for iteration? :-)
Isn't it strange if the PLs of the teams use
such "waterfall" approach to schedule and plan resources their stands are supposed to be iterative ? Are they confused :-) or they have
some good project management reasons (that I'm unaware) to do so ?


Shubhrajit
Kodo Tan
Ranch Hand

Joined: Aug 14, 2001
Posts: 105
quote:
beg to differ...
I understand that most project managers plan their project in such a way that it looks like they are following a waterfall model... but they may be not !!!
Why I feel so...
1. Requirements definitely should not iterate.
2. Analysis also in my opinion should not iterate. Anyway it should not iterate for 90% of projects as it is directly derivable from requirements.

This is always the ideal case. In most of today's ecommerce projects, it's quite impossible not to change user requirements frequently.
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112
Originally posted by Shubhrajit 1. Requirements definitely should not iterate.


I strongly disagree. If you scotch changing requirements, you will prevent yourself from

- being able to react to changing business environments (after all, most software is developed to change the environment it is supposed to used in)

- using your learning experience of developing and using early versions of the product to improve the final product

IMO a much better strategy seems to be to "embrace change" - being prepared for changing requirements.


2. Analysis also in my opinion should not iterate. Anyway it should not iterate for 90% of projects as it is directly derivable from requirements.


As you might guess, I am disagreeing with this, too...


3. Design can iterate. However a competent SW firm will often freeze the design based on past experience and it is highly unlikely that it would change and thus iteration will not be necessary.

4. Construction will definitely iterate. This is a design principle we almost always follow. This is called a build cycle in the construction phase for iterative processes, and there will be multiple build cycles in the construction phase. In other words a construction phase can be said to constitute of multiple sub phases called build cycles...


I don't think it is a good choice to disunite what you call "designing" and "constructing". IMO coding is just designing at a finer scale (the "constructing" is in fact done fully automated by the compiler) and only by coding I get valuable feedback about the applicability of my larger scale design.

5. Integration and System testing should happen only after all iterations ie all build cycles are over.

So why wouldn't you want to get the very valuable feedback of automated tests from early on???


The soul is dyed the color of its thoughts. Think only on those things that are in line with your principles and can bear the light of day. The content of your character is your choice. Day by day, what you do is who you become. Your integrity is your destiny - it is the light that guides your way. - Heraclitus
Shubhrajit Chatterjee
Ranch Hand

Joined: Aug 23, 2001
Posts: 356

Originally posted by Shubhrajit 1. Requirements definitely should not iterate.
I strongly disagree. If you scotch changing requirements, you will prevent yourself from
- being able to react to changing business environments (after all, most software is developed to change the environment it is supposed to used in)
- using your learning experience of developing and using early versions of the product to improve the final product
IMO a much better strategy seems to be to "embrace change" - being prepared for changing requirements.

2. Analysis also in my opinion should not iterate. Anyway it should not iterate for 90% of projects as it is directly derivable from requirements.

As you might guess, I am disagreeing with this, too...

Well, it was my opinion. I think if you do not freeze the requirements early on ... heaven help you on your schedule... even if you are working on a time and material basis ... how can you cope for the end if requirements come at the end of th e construction phase... AND heaven help you if you are on a fixed price contract...
And same applies to analysis...

3. Design can iterate. However a competent SW firm will often freeze the design based on past experience and it is highly unlikely that it would change and thus iteration will not be necessary.
4. Construction will definitely iterate. This is a design principle we almost always follow. This is called a build cycle in the construction phase for iterative processes, and there will be multiple build cycles in the construction phase. In other words a construction phase can be said to constitute of multiple sub phases called build cycles...
I don't think it is a good choice to disunite what you call "designing" and "constructing". IMO coding is just designing at a finer scale (the "constructing" is in fact done fully automated by the compiler) and only by coding I get valuable feedback about the applicability of my larger scale design.

I feel design and construction are quite different. You design your business workflow, user interface (if applicable ) and data modelling using techniques like E-R diagram , etc... and construction refers to actually implementing them using programming languages. I would say design is a planning premise while construction is its execution

5. Integration and System testing should happen only after all iterations ie all build cycles are over.
So why wouldn't you want to get the very valuable feedback of automated tests from early on???

When you construct you will definitely test your components or programs... in Software Engineering paradigm it is called unit testing...
Now once you have your programs ready, you will integrate them together. Testing how they behave in relation to each other is known as integration testing.
Once you have put togther your system, it might interact with other programs outside the boundary of your system. Testing this interaction is system testing.
Ya... you can do integration and system testing within your build cycle. However it might be a wastage of effort.
Shubhrajit
[ February 19, 2002: Message edited by: Shubhrajit Chatterjee ]
Doug Wang
Ranch Hand

Joined: Oct 05, 2001
Posts: 445
Originally posted by Shubhrajit Chatterjee:
1. Requirements definitely should not iterate.
2. Analysis also in my opinion should not iterate. Anyway it should not iterate for 90% of projects as it is directly derivable from requirements.

Dont speculate that requirements never change. You will fasten yourself and be of pretty slow response to business changes.


Creativity is allowing yourself to make mistakes. Art is knowing which ones to keep
Shubhrajit Chatterjee
Ranch Hand

Joined: Aug 23, 2001
Posts: 356
Originally posted by Doug Wang:

Dont speculate that requirements never change. You will fasten yourself and be of pretty slow response to business changes.


I never meant that requirement should not change. I am not that rigid... However this is a project management issue.
I wanted to mean that there should be a requirement control in your project. You will have to freeze your requirement early on. After the requirement is frozen and then there is a new requirement, then it should be taken as a scope change from project management perspective. This would result in a new plan for the project.
Why should you want to freeze your requirement ?
1. Suppose you are working on a fixed price contract for your client. Say about a week remains for final delivery when some person comes with a changed requirement which might require a change in the database design... how are you going to cope with it ?
2. You are slightly at ease when you are at a time and material contract. Still, you have to give a time estimate to your client. Now if these kind of requirement changes take place often, you will have a huge schedule slippage, which would not look good from a project management point of view.
Software companies which are at the level 5 of Competency Maturity Model behave in this way. They have defined processes which freeze the requirements at an early stage and any further change is taken as a change of scope.
One last word... what is a requirement change ? If you have radio buttons in your HTML page and the client wants Combo Box ... few people would call it a change in requirement... but if the client wants a new HTML form with associated back end processes ... that is what I would call a requirement change
Ilja Preuss
author
Sheriff

Joined: Jul 11, 2001
Posts: 14112

I wanted to mean that there should be a requirement control in your project.

Definitely!
You will have to freeze your requirement early on.

If you mean by "freeze" "don't change without consideration of impact on the release plan", I agree. If you mean something else, please elaborate.

After the requirement is frozen and then there is a new requirement, then it should be taken as a scope change from project management perspective. This would result in a new plan for the project.

I agree, though I would probably rather call it an adjusted plan.

Why should you want to freeze your requirement ?
1. Suppose you are working on a fixed price contract for your client. Say about a week remains for final delivery when some person comes with a changed requirement which might require a change in the database design... how are you going to cope with it ?

Estimating the cost of the requirement. Giving the customer the choice to either
- sacrifice requirements of lower priority and equal cost for the new requirement,
- extend the contract (including the price) to include the new requirement or
- resign the new requirement for now.

2. You are slightly at ease when you are at a time and material contract. Still, you have to give a time estimate to your client. Now if these kind of requirement changes take place often, you will have a huge schedule slippage, which would not look good from a project management point of view.

You could use the same strategy as presented above.
For me it seems to be critical to make visible the impact of new requirements to the stakeholders. They always should know where the project stands.
What do you think?
Gerry Giese
Ranch Hand

Joined: Aug 02, 2001
Posts: 247
I think Ilja has it right.
You don't necessarily iterate your requirements during "normal" flow of the project's process. If the "customer" happens to need a change that impacts the requirements, then an iteration is needed over the requirements. You shouldn't let your customer change things after agreement is already made or you've got a moving target. The idea is to write the "contract" in such a way that the customer must re-open negotations if they do need a change in the requirements.
I *DO* think that analysis iterations are a *MUST*. My concept of analysis is that I have the requirements, and I am analyzing them to understand their meaning and how it impacts the project. For example, a requirement in a web-based retailer application is that appropriate taxes are applied to the sale. You will obviously spend a fair amount of time in analysis for this requirement, but a) you don't necessarily need to have 100% of your analysis done before you feed your documentation into the next phase(s), and b) as you enter design and construction phases, you may discover nuances and details that need clarification and additional analysis. Either way, you need another iteration to flesh out the analysis. Once you've done an two or three iterations, you obviously should be pretty darn near complete.
What the iterations give you is an way to inject flexibility and fluidity into the project to allow for change, improved understanding of the problem and solution, and as a side benefit, and increased amount of parallelism in tasks. I will note that the benefits to iterations are greater the larger the project, and small projects are likely to not need but 1 or 2 iterations in analysis and design. But iterations *are* useful, except when you're trying to do your schedule!
shailesh sonavadekar
Ranch Hand

Joined: Oct 12, 2000
Posts: 1874
when you are very , very sure about the requirements & you executed the similar kinds of projects then it is said that waterfall is the best methodology.
but , you are not sure about the requirements , then iterative development should be good .
Joe Gilvary
Ranch Hand

Joined: May 11, 2001
Posts: 152
I think one of the main points behind agile
processes is that most software development is done
for business purposes, in a rapidly changing environment,
to meet a set of requirements which typically change
over short periods of time. Since those changes can
occur during the development cycle, it makes sense to
design your process to cope with on-going change as
best you can.
Software development as a component of the
business world needs direction aligned with the needs
of the business. And those needs can change markedly,
in a very short time.
You usually are not very, very sure about requirements
unless you are simply replacing an existing system
with one that is functionally identical. That is not a
common type of project in the business world today.
With Java available on IBM big iron, have you
heard of many COBOL to Java porting projects which
require an exact match of existing functionality?
Even in that case, how likely is it that the requirements
(existing functionality) are known and captured in
document form?
My point is that the emergence of sw development approaches
like XP is a direct result of the desire to align the
sw development process with the needs of the business
processes which drive it.
Thanks,
Joe
[ February 20, 2002: Message edited by: Joe Gilvary ]
[ February 20, 2002: Message edited by: Joe Gilvary ]
Shubhrajit Chatterjee
Ranch Hand

Joined: Aug 23, 2001
Posts: 356
Well, it seems most of the people is disagreeing with me
1. Someone pointed out that business requirement changes drastically, during the life cycle of the project ... is this really true ?
Well, technology is taking great strides . However, the core business processes are still very much the same ... at least my experience says so ...
Large companies still use the COBOL programs they have developed since 1960. Right now , they often discuss about the usability of the system ... they are now building good GUI front ends. The employees do not like the green and white CICS screens: they want a web interface. That is why middleware products like MQSeries are a huge success.
I currently work in one such project... web enabling of a M/F CICS - COBOL application using MQSeries. The back end business processes are still the same. We only built a jazzy front end to it.
The SW we are developing is currently in the 7th release... after every release we get some new requirements , prepare a new contract, a new project plan ... and of we go ...
The process we use is one defined by our company ... our company is at level 5 of CMM and we believe that we are mature enough to define our own processes.
Going back to the original arguement ... since our project is in the 7th release, requirements have been added 6 times to the original requirement. But analyzing the requirements we found that the changes they required were mainly at the Usability level, the 30 back end COBOL programs still remained the same. What we were doing is adding easy navigations between programs, short cuts , keys , nice looking forms which they can print and use as reference , etc...

We are in this development for almost 2 years. Initially we were on a fixed price contract but now we operate on a Time and Material basis. If we had not frozen the contract before every release of the product ... imagine what would happen to the contract.
The way we are managing the project makes everyone happy. We are happy as we are generating a lot of revenue and the client is happy as well they are using a fully functional product.
Joe Gilvary
Ranch Hand

Joined: May 11, 2001
Posts: 152
Drastically is a strong word. If the requirements
seem to change drastically during the development
process, I think it is more likely that the requirements
were never captured correctly in the first place.
If your project is replacing the CLI user interface with
a GUI interface, and the core business process is
not changing, you do have a set of requirements which
might be very well known. If the interface now needs
to be extended to expose an API to those same COBOL
programs to one or more of your client's business
partners through an extranet, I dare say that you
will see how "requirements" can change
during a project life cycle.
Thanks,
Joe
Tiger Scott
Ranch Hand

Joined: Mar 01, 2001
Posts: 223
Shubhrajit :
1. I think requirements do change- frequently and drastically often. The reason could be as simple as oversight/some manger forgot to give input OR something like the change in business environment.
2. From your project description it appears you are working on providing a front end to a legacy system- so the leagcy system is a constant- you 'change' the requirements for the scope of the project only.
3. In a fixed price contract everbody is stuck with the requirements- sometimes it leads to not so happy clients.
$.02
Sanjay
Mark Herschberg
Sheriff

Joined: Dec 04, 2000
Posts: 6037
Originally posted by Sanjay Bahal:
Shubhrajit :
1. I think requirements do change- frequently and drastically often. The reason could be as simple as oversight/some manger forgot to give input OR something like the change in business environment.

Very often it's a change analogous to the Heisenberg Uncertainty Principle. The act of observation perturbs the system. The description my HS chemistry teacher gave is in order to measure the temperature of the turket in the oven, you need to open the door to look at the thermometer. Of course, this then changes the temperature in the oven.
On a larger scale, we see public work projects, like the Big Dig here in Boston. The Big Dig is a $14 billion (yes, billion) project redoing the main traffic route into downtown Boston. It's been going on for almost 10 years now. (We're ignore the fact that the time and schudule has been off by more then a factor of 2.) Its supposed to make it easier for workers to dirve into Boston. Of course, during these last 10 years, housing patterns have begun to change in response to it. If people didn't understand the system well, its possible that despite making a wider roadway, enough people moved, hoping to use the new system, that it ends up no less congested then before.
The same is true with software. Most customers don't fully understand what they want or how exactly it will work. Not because they're stupic, but because most software is complex. Usually the first time you shwo them what they're getting (during an alpha or beta release), they have a better idea what it is they'll get; this changes how they think, which in turn changes their requirements.

--Mark

PS I was a physics major, so yes, I know this is not technically the same thing as the Uncertainty Principle, but it's along the same vein.
John Wetherbie
Rancher

Joined: Apr 05, 2000
Posts: 1449
Good discussion! Sorry I arrived so late but just a couple points.
The best (and maybe the shortest) statement I have seen regarding software requirements is by Kent Beck in eXtreme Programming explained: "The development of a piece of software changes its own requirements." So you'll definitely need to manage your requirements and control changes to them (in the fashion of your chosen approach) because requirements (what the customers needs) are iterative to a certain extent.
One thing about waterfall. I have read that the person who originally came up with what is known as the waterfall method originally intended it to be used in an iterative fashion not as one big, long serialized process. The paper this was in was written by Robert Martin (of Extreme Programming in Practice). Has anyone else seen any references to the contention that the way waterfall is current practiced is not what the originator intended?
Sorry if I've winged off on a tangent...
John


The only reason for time is so that everything doesn't happen all at once.
- Buckaroo Banzai
John Wetherbie
Rancher

Joined: Apr 05, 2000
Posts: 1449
This may be way too late but going back to the post that started this thread...
What is described there is what we all know (and love/hate/tolerate?) as the waterfall. I have been on projects where management has said a schedule like that is "iterative" or "incremental" but just because you call a dog's tail a leg doesn't actually make it a leg.
Iterative projects I have been on do have some overhead before you jump into the iterations but there ARE iterations that have some analysis, design, CUT, and deployment! Not having iterations in what is supposed to be an iterative project isn't a problem with a tool (Though I could list lots of things I think are wrong with MS Project! Another discussion...) it is a problem with how the people are thinking about the project's structure.
John
Doug Wang
Ranch Hand

Joined: Oct 05, 2001
Posts: 445
Hi, all
Is "iterative" equivalent to "incremental"? My answer is yes, but people here dont think so.
John Wetherbie
Rancher

Joined: Apr 05, 2000
Posts: 1449
I think in common usage the two are used interchangeably, possibly incorrectly. However, in articles that predate "agile" the term or phrase I remember is "iterative and incremental". Iterative because you iterate over a sequence of steps (analysis, design, CUT, deploy) and incremental because you are building up the system functionality, well, incrementally. So an iterative process is probably not equivalent to an iterative process.
Based on the exam question and the answers I would go with A as the answer.
Other views on this?
John
Jim Berg
Greenhorn

Joined: Feb 21, 2002
Posts: 4
I would take exception to "Requirements" being defined as a monolothic fixed structure. If you look at any project during a coding phase, there is design work and requirements changes being done by every programmer. There are architecture and infrastructure requirements and designs that need to be performed up front and should be set at a high enough level that they can remain fixed through out the project. But don't think for a minute that means that the requirements and design are fixed. Programming is like a conversation, the programmer codes an algorithm and the system responds. As a result of the response, the programmer changes the code or moves on to the next code segment. There is "communication" after a fashion going on here. And the nature of that communication results in movement towards a final design.
 
It is sorta covered in the JavaRanch Style Guide.
 
subject: Iterative or waterfall ?