One of the key parts of Agile Software Development is the concept of iterative and incremental development (IID). I here people say this frequently – but when I ask more specific questions I hear things like, “We we develop small chunks and we have two week iterations.” We there is an entire group of Agile methods becoming mainstream that are iteration-less – instead they follow a continuous flow model. So if Agile requires IID, and Lean and Kanban are iteration-less, does that mean they aren’t Agile. Well, no, because iterative development doesn’t mean you are doing iterations.
In incremental development various parts of the system are developed at different times or rates, and integrated as they are completed. You can do this in a waterfall project or an iterative project. The alternative to incremental development is to develop the entire system with a “big bang” integration.
In iterative development the team plans to iterate, or go back over parts of the system, to revise and improve the system. In iterative development testing and/or user feedback is used to revise the targets of the successive deliverables. You don’t have to have iterations to be doing iterative development. The practice of iterations arises from a desire to coordinate feedback from increments with revising a future deliverable. Iterative and Incremental Development work well together. A hard stop, or a time-boxed iteration, has historically been useful to coordinate feedback with deliverables.
One of the powerful and interesting things that has happened in the last decade in software development is Continuous Integration. This is a practice that arose from Martin Fowler and Kent Beck in the Extreme Programming community. Continuous Integration means we can deliver increments of software at any time. I have worked with a team whose continuous integration environment was so robust they were prepared to responsibly deliver updates to production several times a day. When you can continually deliver working software then you can eliminate the hard stop. Just perform the user feedback of the application whenever it makes sense. You can use this feedback to revise the successive deliverables without creating a hard top for everyone on the team.
No iterations in IID?
There are several benefits of going iteration-less. A big benefit of single piece flow is not stopping the entire team every week or two weeks. Reducing batching to a single item results in an increase in the amount of time development can be focused on writing code. For example, if the entire team has to stop for two days (one for delivery one for planning) every two weeks, you are effectively eliminating 20% of the time developers can work. A second benefit of going iteration-less is the elimination of an artificial constraint that an item must be small enough to fit into the time-box. While it is always better to deliver small pieces at a time – value cannot always be shaped into the time-box. A third benefit of going iteration-less is that demand on related resources has a smoother pattern. Time-boxes batch related work such as testing, requirements grooming, and deployment. The pace is more sustainable for the non-development resources.
There are several drawbacks to going iteration-less if the team doesn’t address them. First there can be challenges with coordinating feedback into successive deliverables. If the developers just keep writing code – when do we update the backlog with the input we would historically receive at the iteration review. Also, since there is no time-box, there is also a perceived lack of pressure to get to done. Additionally, by not enforcing effective decomposition there can be a lack of pressure to design well. Also, there isn’t a fixed time to stop and review the development processes to keep the team in synch. Finally, continuous delivery can result in continuous demand on related resources like analysts and quality assurance.
There are effective strategies for dealing with each of these drawbacks. For example, the developers that just delivered a feature can stop to get feedback from the customer without disrupting the entire team. You can create visibility into delays by tracking the days of effort against each story as it moves across the board – which will result in pressure to get to done. Enforcing norms that work must be small enough to get through in less than two weeks – but allowing for exceptions when it makes sense to the team will help ensure good design. You don’t have to come to a hard stop on everything in process to do a team retrospective (operations review). Finally, the related resources timing can be decoupled from the teams cadence. For example, business analysts can still groom backlog in weekly batches to create some buffer.
History if IID
One of the interesting things for many current agilists to understand is that we have been doing IID in software development since the late 50’s – that’s right – half a century. Here are some highlights.
1957: Gerald Weinberg runs a project at IBM using IID practices.
1960: Gerald Weinberg starts teaching IID at the IBM System’s Research Intitute
1969: Robert Glass published a paper on current practices in IID in ACM Computer Surveys
1975: Vic Gasill and Albert Turner publish Iterative Enhancement: A Practical Technique for Software Deveopment in IEEE Transactions on Software Engineering
1980: Gerald Weinberg publishes Adaptive Programming: The New Religion
1984: Tom Gilb recommends two week iterations as a best practice
1985: Barry Boehm promotes the spiral model – using risk as the key prioritization method
1986: Fred Brooks publishes No Silver Bullet which articulates the benefits of IID over waterfall
1988: Tom Gilb publishes Principles of Software Engineering Management showing an evolutionary approach to development
1990’s: Scrum, DSDM, RUP, XP, and FDD all become mature practices
It is just in this decade, fueled by improved development tools, lower cost computing and network, and in response to pressure to rapidly deliver solutions with a high level of unknowns that IID, in the form of Agile methods, has become common place.