Iterative and Incremental Development
One of the key parts of Agile Software Development is the concept of iterative and incremental development (IID). I here people say this frequently – but when I ask more specific questions I hear things like, “We we develop small chunks and we have two week iterations.” We there is an entire group of Agile methods becoming mainstream that are iteration-less – instead they follow a continuous flow model. So if Agile requires IID, and Lean and Kanban are iteration-less, does that mean they aren’t Agile. Well, no, because iterative development doesn’t mean you are doing iterations.
In incremental development various parts of the system are developed at different times or rates, and integrated as they are completed. You can do this in a waterfall project or an iterative project. The alternative to incremental development is to develop the entire system with a “big bang” integration.
In iterative development the team plans to iterate, or go back over parts of the system, to revise and improve the system. In iterative development testing and/or user feedback is used to revise the targets of the successive deliverables. You don’t have to have iterations to be doing iterative development. The practice of iterations arises from a desire to coordinate feedback from increments with revising a future deliverable. Iterative and Incremental Development work well together. A hard stop, or a time-boxed iteration, has historically been useful to coordinate feedback with deliverables.
One of the powerful and interesting things that has happened in the last decade in software development is Continuous Integration. This is a practice that arose from Martin Fowler and Kent Beck in the Extreme Programming community. Continuous Integration means we can deliver increments of software at any time. I have worked with a team whose continuous integration environment was so robust they were prepared to responsibly deliver updates to production several times a day. When you can continually deliver working software then you can eliminate the hard stop. Just perform the user feedback of the application whenever it makes sense. You can use this feedback to revise the successive deliverables without creating a hard top for everyone on the team.
No iterations in IID?
There are several benefits of going iteration-less. A big benefit of single piece flow is not stopping the entire team every week or two weeks. Reducing batching to a single item results in an increase in the amount of time development can be focused on writing code. For example, if the entire team has to stop for two days (one for delivery one for planning) every two weeks, you are effectively eliminating 20% of the time developers can work. A second benefit of going iteration-less is the elimination of an artificial constraint that an item must be small enough to fit into the time-box. While it is always better to deliver small pieces at a time – value cannot always be shaped into the time-box. A third benefit of going iteration-less is that demand on related resources has a smoother pattern. Time-boxes batch related work such as testing, requirements grooming, and deployment. The pace is more sustainable for the non-development resources.
There are several drawbacks to going iteration-less if the team doesn’t address them. First there can be challenges with coordinating feedback into successive deliverables. If the developers just keep writing code – when do we update the backlog with the input we would historically receive at the iteration review. Also, since there is no time-box, there is also a perceived lack of pressure to get to done. Additionally, by not enforcing effective decomposition there can be a lack of pressure to design well. Also, there isn’t a fixed time to stop and review the development processes to keep the team in synch. Finally, continuous delivery can result in continuous demand on related resources like analysts and quality assurance.
There are effective strategies for dealing with each of these drawbacks. For example, the developers that just delivered a feature can stop to get feedback from the customer without disrupting the entire team. You can create visibility into delays by tracking the days of effort against each story as it moves across the board – which will result in pressure to get to done. Enforcing norms that work must be small enough to get through in less than two weeks – but allowing for exceptions when it makes sense to the team will help ensure good design. You don’t have to come to a hard stop on everything in process to do a team retrospective (operations review). Finally, the related resources timing can be decoupled from the teams cadence. For example, business analysts can still groom backlog in weekly batches to create some buffer.
History if IID
One of the interesting things for many current agilists to understand is that we have been doing IID in software development since the late 50’s – that’s right – half a century. Here are some highlights.
1957: Gerald Weinberg runs a project at IBM using IID practices.
1960: Gerald Weinberg starts teaching IID at the IBM System’s Research Intitute
1969: Robert Glass published a paper on current practices in IID in ACM Computer Surveys
1975: Vic Gasill and Albert Turner publish Iterative Enhancement: A Practical Technique for Software Deveopment in IEEE Transactions on Software Engineering
1980: Gerald Weinberg publishes Adaptive Programming: The New Religion
1984: Tom Gilb recommends two week iterations as a best practice
1985: Barry Boehm promotes the spiral model – using risk as the key prioritization method
1986: Fred Brooks publishes No Silver Bullet which articulates the benefits of IID over waterfall
1988: Tom Gilb publishes Principles of Software Engineering Management showing an evolutionary approach to development
1990’s: Scrum, DSDM, RUP, XP, and FDD all become mature practices
It is just in this decade, fueled by improved development tools, lower cost computing and network, and in response to pressure to rapidly deliver solutions with a high level of unknowns that IID, in the form of Agile methods, has become common place.
Dennis,
I wish you would give us some context around when iterations might be beneficial and when they might be less beneficial. Having this discussion outside of a contextual framework is somewhat counterproductive. There are few absolutes and much of this has a time and a place where it is most effective.
Iterations are not always waste and continuous flow is not always the best approach for a team. Establishing not only what works, but when to apply it, will help lead the community forward. Otherwise, we have people trying stuff just because it worked somewhere, for somebody, regardless if it is a good fit or not.
Mike
Great point Mike.
I think there are some clues to when to use iterations vs continuous flow in the benefits and drawbacks of continuous flow. In fact, I have seen situations where in a multi-team project where you should have iterations on one team and continuous flow on another.
I will address this in my next post.
Dennis Stevens
Mike,
I have a post addressing this on Monday morning.
Dennis
Pretty good but I have some corrections and deeper points. But just boarding plane to Estonia! Larman has researched history. And Southampton u. I’ll send it. I am earlier than you cite! You forget the important Evolutionary category. Defined as using feedback at each cycle. Your ii definitions are at odds with others such as DOD. More later
tom gilb. Oslo.
Thanks Tom,
I look forward to the input.
I didn’t spend time in this post talking about how feedback at all levels (product, process, and organizational) at each cycle is the key point that distinguishes Agile from IID. I talk about it a little more in my next post. Feedback and your pioneering work with Evolutionary Project Management is important.
I will read what you send, update the timeline above, and clarify my thoughts on feedback and the ii definitions in a post next week. Enjoy Estonia.
Take care,
Dennis
[...] my last few posts (here and here), I have been talking about Iterative and Incremental Development (IID). This is where the [...]