Here is the presentation I did at the Lean Software and Systems Conference in Atlanta. We combined Strategy Thematic Goal Models, Capability Mapping, Kanban at the strategy execution level with a cascading Kanban at the development level. The presentation was well received.
Last week I wrote a post on Iterative and Incremental development. In that post I pointed out that you can do iterative and incremental development without having time-boxed iterations. Mike Cottmeyer asked in a comment to clarify when time-boxed iterations made the most sense and when continuous flow made the most sense. This post answers that question.
When Time-boxed Iterations Make the Most Sense
Time-boxed iterations serve several purposes that are not well served in continuous flow. They create an opportunity for the team to stop and get feedback from the customer on their product. They can help with chunking up work to support the planning process and validation of the design of the solution. There are times when this is really important.
So, time-boxed iterations work best:
1. When there is a lot of opportunity for feedback on current deliverables to significantly update subsequent work
2. When the planning horizon is stable enough that a subset of the backlog can be prioritized for at least the duration of the iteration
3. When the backlog items can fit in the iteration time-box
4. When dependent resources (for example, planning and testing) can not be available to support continuous flow
For example, time-boxed iterations work best at the outset with a new product, new technology, or substantially new set of features.
When Continuous Flow Makes the Most Sense
Continuous flow has some advantages over time-boxed iterations. Overtime, the level of resources available to do work is higher and the backlog is much more flexible and adaptive to variation in the arrival of work. For example, once the product architecture is stable and well defined and work is arriving in bursts with various levels of urgency.
So, continuous flow works best:
1. When the work is arriving continuously
2. When the planning window can be (or must be) very small
3. When the outcome is pretty clear and feedback is not as likely to impact subsequent work
4. When there are multiple work streams operating at different cadences
For example, continuous flow when working on defects and small enhancements in a stable product or working with technical support issues.
Non-relevant factors
Effective planning. Data can be gathered from both iterations (velocity) and continuous flow (lead time) to determine when a product can be delivered and to establish visibility into the current status of the project.
Retrospectives. Iterations create a natural break for retrospectives. There is no reason why retrospectives must be tied to iteration releases. Continuous flow teams can have retrospectives on either an event driven or calendar driven basis.
Motivation to get to done. Iterations create pressure to complete work in a specific time-box. Continuous flow creates pressure to met lead-time commitments (policies).
How they can work together
We are making a standard practice of combining Kanban at the strategic planning level to support continuous flow of strategic initiatives across the organization and Scrum (or Kanban) at the development team level. It doesn’t make sense to us to create time-boxed deliverables at the strategy execution deliverable level. We generate plenty of pressure to move forward from the time commitments on the tasks on the Kanban board. At the development level – when there is a lot of opportunity for feedback to update subsequent work – we (may) use iterations to keep the team focused. For defects and minor enhancements we continue to use a continuous flow model – maybe within the same team.
Iterations versus Continuous Flow?
I believe this is a hammer versus saw discussion. Each approach is suited to fit different scheduling and feedback needs. In many cases, time-boxed iterations make a lot of sense but would result in a less than optimized approach then in other cases. I believe that early on in a product (when there is a lot to be learned and a lot of risk to manage) iterations probably make the most sense. At some point, when the product has switched primarily to support, continuous flow makes the most sense. I hope this answers Mike’s question from my previous post.
I have been reading a lot about of the recent group discussions and papers regarding Kanban and Lean Software Development. One of the things I have noticed is that there is not a consistent application of the concepts regarding time. One of the reasons we establish a body of knowledge is to create a common language that helps us communicate based on a shared understanding. This inconsistency in the use of terms around time create a gap in understanding. In an effort to help the conversation here are the appropriate definitions for each type of time.
Durations
Duration is simply a measure of elapsed time.
Lead Time: The average time it takes for one unit to go through the entire process – from start to finish – including time waiting between sub-processes. Lead time can take different perspectives.
Development Lead Time is the time it takes for a unit to go through development from release from the backlog until it is done (which includes analysis, development, test, acceptance, deployment – but maybe not release to production). It is interesting to the development manager.
Order Lead Time is the time that elapses between when the customer places an order and when the customer receives what they ordered. In cases where requests are queued in a back log and releases are decoupled from development completion this can be far longer than the Development Lead Time. This is interesting to customer.
Order to Cash Lead Time is the time from when the customer places the order to the time cash is received by the business. While this is often longer than the Order Lead Time it can be shorter than the Order Lead Time when the customer prepays. This is interesting to the person paying the bills.
Process Lead Time: The time the unit is being worked on in a specific process before it is passed on to the next process. For example, within Development Lead Time this could be analysis, development, automated test, acceptance test or deployment time. This is interesting to the person performing the specific process or activity.
Processing Time: The time the unit is actually being worked on. It would be the sum of all the Process Lead Times. This is interesting to use as we try to improve our Lead Time.
Value Add Time: The Processing Time spent on a unit that transforms the unit in a way that the customer is willing to pay for. Not all Processing Time is Value Add Time – although non-Value Add Time can be necessary due to compliance requirements, to optimize performance of value add resources, and to minimize rework. This is interesting as it has been the subject of much of the lightweight process effort in Agile development. When taken to far it results in increased Rework Time and increased Process Lead Time.
Rework Time: The Processing Time that results from poor quality. Failure Demand is the amount of resource that is consumed by Rework Time. This is interesting as it is expensive.
Queue Time: The time a unit sits between sub-processes waiting for someone to work on it. This is interesting to us as we try to improve our Lead Time. This is no cost to improving flow to reduce Queue Time but it reduces Lead Time.
Cadences
Cadence is a measure of time per unit.
Cycle Time: The average time between units coming out the end of the pipe. This is interesting because it helps us predict Lead Time.
Takt Time: How often units need to come out the end of the pipe to meet customer demand. This is interesting because it reflects the rate we need to produce to optimize investment against revenue.
You would expect Lead Time to be the sum of Processing Time, Queue Time, and Rework Time – and it is in a purely serial process flow. But in cases where work moves in parallel the sum of Processing Time, Queue Time and Rework Time will be greater than the Lead Time. Lead Time is interesting because it is what we use to make commitments. Improving Lead Time can be done by reducing Queue Time, shortening Process Lead Time, eliminating Rework Time, minimizing non-Value Add Time, and performing processes in parallel.
The concept of Value Add Time is interesting and has had strong impact on Agile processes. At some point things like documentation, planning, testing, and adding additional features are not adding value in a way the customer is willing to pay for. Reducing these items to the bare minimum time that is necessary to meet the needs of the customer is helpful. Reducing them to the point they increase rework or reduce the productivity of the value added performers is not helpful.
The biggest inconsistency I have noticed is the improper use of Lead Time (a duration), Cycle Time (a cadence), and Takt Time (a cadence). In summary, Lead Time is how long it takes us to build and deliver it, Cycle Time is how often a new one is produced, and Takt Time is the rate of customer demand. In a stable system, Duration and Cadence are related by Little’s Law. The cadence of Cycle Time will equal WIP divided by the duration of Lead Time.
I believe that consistent use of these terms will help us have more meaningful conversations and continue to improve the ways that we deliver software to our customers. Part of my focus over the last decade has been on reducing Lead Time through reducing WIP, Queue Time, and non-Value Add Time, improving quality to reduce Rework Time, and improving architecture, automation, and shared understanding to reduce Processing Time. Having a shared understanding for each type of time leads to productive conversations around specific techniques and improves the teams ability to focus on the right types of improvements.
There continues to be discussion about Scrum (as a brand) vs. Kanban (capital K) vs. Crystal vs. FDD, etc. If fact, I received a comment today on a March post discussing the Scrumbut test. My comment became to long and is so relevant to recent discussions that I turned it into a blog post. From my perspective these method wars tend to fall into “How Trap” discussions. How Trap discussions are not about specific outcomes you are trying to achieve and developing a situationally specific approach to achieving those outcomes.
Take a look at the Scrumbut test I learned about from Jeff Sutherland. Whether you are doing (or even like) Scrum – when viewed through an outcome lenses, these questions lead to discussion regarding clearly important outcomes. Below, I list the question category from the Scrumbut test, then the outcome that is desired from the practices described in the Scrumbut test, then a hopefully clarifying description. The practices are detailed at the Scrumbut link.
Question 1 – Iterations: Limit Work in Process – don’t start anything you can’t finish
Question 2 – Testing within the Sprint: Maintain Code Quality – Shorten quality feedback loops to zero to maintain code quality and reduce the waste of rework
Question 3 – Agile Specification: Communicate Business Needs to Development – Ensure developers understand outcomes while minimizing coordination and transaction costs, maintain traceability to business need
Question 4 – Product Owner: Maintain Product Roadmap – The business communicates a plan of what is being built
Question 5 – Product Backlog: Prioritize Investments Based on Return – the business prioritizes work based on a current understanding of business value
Question 6 – Estimates: Provide Meaningful Effort Estimates – Understand rate the team can produce work to support planning, investment decisions, and customer commitments
Question 7 – Burndown Chart: Communicate Release Schedule – Be able to predict content and timing of future releases. This clarifies corrective action, next most valuable investment decisions, marketing decisions and customer commitments
Question 8 – Team Disruption: Maintain Productive Work Environment – Management understands and supports the focus on rapid delivery. Trust is established that the team will be able to meet current and future commitments.
Question 9 – Team: Develop an Empowered Team – The team feels empowered to make decisions about how to move forward. Management has provided sufficient guidance and direction that they trust the team will make operational and tactical decisions aligned with the best interest of the business.
Yes, Jeff has a brand he is protecting. If you think you are doing Scrum and you aren’t achieving these outcomes then you aren’t doing what was intended. Bad Scrum and Scrumbut hurts the Scrum brand. In fact, since Scrum is a dominant Agile brand, it hurts all of us trying to move Enterprises towards improved software agililty.
But, Scrum isn’t the only way to achieve these outcomes. I can map these outcomes to mature implementations of XP, DSDM, FDD, Crystal and Kanban (etc.). Now the interesting conversations sound like –
The outcomes described in the Scrumbut test can be achieved by applying specific strategies that are consistent within an organization’s environment. When these outcomes are adopted development teams tend to become more mature, better at delivering software (many times dramatically), and improve their work environment. Regardless of method, you need to be focused on the outcomes that result in improvements in productivity and find the way that works in your organization to adopt and sustain the outcomes.
I really enjoyed Agile 2009 this year. There are a lot of really smart people working on this space of learning to better ways to develop software and helping others to do it. Thanks to everyone who saw my presentation, Feeding the Agile Beast. I would be happy to send you a copy and a simple version of the Excel tool I’ve used to prioritize and scope work for Agile/Kanban development teams.
Alistair Cockburn, in his opening keynote, I am here to Bury Agile not to Praise it, pointed out that we more or less know how to develop software in small teams. Agile practices have, to some extent, been absorbed into the way that software is developed. What are we doing now is figuring out how to drive even more value from the enterprise from our ability to build software.
Here are some of the problems people are trying to solve that I found extremely interesting.
Agile Project Management. There were a number of talks on Agile Project Management. Mike Cottmeyer presented the Agile PMP. Jim Highsmith presented Agile Project Management. Thoughtworks sponsored a kick off meeting led by Jesse Fewell for PMI’s new PMIAgile Virtual Community. The event was attended by the CIO of PMI as well as Martin Fowler, Alistair Cockburn, Ward Cunningham, Jim Highsmith. It turns out that PMI has been doing Agile development for over two years on their internal IT projects. Agile isn’t just for the small team. With effective project management Agile projects can be coordinated across large distributed enterprises.
Agile Business Analysis. No, that isn’t an oxymoron. There were several presentations on using Business Value and Risk to prioritize and scope what goes into development teams. Chris Matts and Olav Maasen presented on Real Options. David Anderson presented on a Risk Management. Julian Evert presented a financial risk based option model to make decisions about what products to work on and what to invest in. These are practioners and coaches using advanced methods of prioritization to make sure the highest value items are getting put into the development teams.
Agile in the Enterprise. Agile is not just for small teams anymore. There were at least ten Enterprise Agile presentations including Dean Leffinwell’s Scaling Software Agility and Ahmed Sidky’s Pragmatically “Crossing the Chasm” from Project-level to Enterprise Adoption. Mike Cottmeyer and I had breakfast with Jim Highsmith to discuss our new book and he is involved in a 40,000 person Agile transition.
Kanban. Kanban is a hot topic this year. We gave out over 300 buttons and stickers at the Freshers Faire on Monday and the I Kanban and limitedwipsociety.org t-shirts were popular items. Chris Shinkle presented Kanban Adoption at Software Engineering Professionals (SEP). Chris has implemented Kanban almost 30 times and has consistently experienced success. There were a number of Open Jam events discussion the Pro’s and Con’s of Kanban including one by David Anderson applying Resource Liquidity. Despite a lack of clarity on many attendees part regarding Kanban – this approach to visible management, continuous process improvement, and establishing continuous flow has legs. David Anderson has an excellent new book coming out in the next six months or so that takes you from basic implementation all the way through advance improvement techniques.
Games as Learning. I wasn’t sure about this one. The Lean Lego Game, The Beer Game with Agile Teams, The Bottleneck Game, The Business Value Game, Agile Cross Culture with Games, The Distributed Agile Game, The Kanban Game, and The Agile Game all used games to rapidly get new and challenging concepts across. The ones by Pascal Van Cauwenberghe and Portia Tung were very well done. I think there will be more game playing and simulation as learning moving into organizations.
In general, the content of the presentations were good. The interaction with experts and practioners was priceless. Agile is clearly moving mainstream and into the Enterprise. New techniques and practices are being developed and practiced. There are some great new ideas being applied in the software development and product development arenas. As we get better at scaling and adoption and we learn to apply emerging development, management, and analysis techniques, we should expect to see great advances in productivity gains and the value created.
In the Agile Manifesto it says, “We are uncovering better ways of developing software by doing it and helping others do it.” I have been paying attention to the conversations between the BUFD traditionalists, the Agilists, and now the Kanban communities. This conversation has not always been friendly – and some members of each community certainly aren’t looking to learn about better ways of developing software. In fact, the contentious debates are creating obstacles to helping others do it.
I find that there are valuable capabilities in each of the practice areas. Agile was never completely different than BUFD. It was 20-30% different. And that difference, when done well in the right context, dramatically improved the value of the software being developed and the velocity that it was delivered. Kanban is probably 10-20% different than Scrum, the most prominent instance of Agile. XP doesn’t really come into the discussion because XP practices will be applied in both of the situation. In the right context, Kanban will deliver dramatically improved value and velocity over explicit Scrum. After all, some companies have moved to single-unit engineer-to-order flow over small batches and realized dramatic improvements.
As I have been wrapping my brain around the distinctions and benefits, I keep coming back to Gerald Nadler’s book from the mid 1990’s, Breakthrough Thinking: The Seven Principles of Creative Problem Solving. Nadler had developed a meta-model for problem solving (solution finding) that has consistently delivered dramatic results. I am going to share the seven principles with you and then use them to talk about why I think the principles in Kanban can deliver results that are better than teams doing Scrum.
1. Uniqueness: Every problem is unique. You can’t directly copy a solution from elsewhere. Only if the context of the purposes shows it is useful should some available solution be considered.
2. Purposes: Always and continually ask the purpose. Then ask the purpose of that purpose. Expand as far as possible with this exercise. Focusing on purpose lets the uniqueness of the situation become clear and strips away nonessential aspects of the problem context.
3. Solution-after-next: Find solutions today that achieve the focus purpose, based on what might be the solution of the future. Having a target solution I the future gives direction to near-term solutions and infuses them with larger purposes.
4. Systems: Employ a format that includes all elements and interrelationships that the target solution will entail. Every solution or system is part of a larger system, and solving one problem inevitably leads to another. Understanding the elements and dimensions that comprise a solution lets you determine in advance the complexities you must incorporate into the implementation of the solution.
5. Limited Information Collection: Before taking the time and wasting the effort to collect and analyze extensive data, determine what purposes would be achieved by gathering the data. Study the solution, not the problem.
6. People Design: Throughout the process of finding a solution, give everyone who will be affected by that solution the opportunity to take part in its development. People are willing to explore purposes and solutions-after-next, rather than to gather data about what is wrong with the current system. Those who carry out and use the solution should be intimately and continuously involved in its development. Also, the proposed solution should include only the critical details, in order to allow flexibility for those who must apply the solution.
7. Betterment Timeline: Even as you design today’s solution, schedule future changes necessary to its continuous success. The only way to preserve the vitality of a solution is to build in and then monitor a program of continual change. A sequence of purpose-directed solutions and knowledge of the solution-after-next are bridges to a better future.
Scrum explicitly calls out several of these principles, particularly uniqueness, purposes, and people design. It provides direction that you should be looking for a way to remove impediments. It doesn’t explicitly provide guidance for doing that and a good, experienced manager is already applying many of the other principles. As I was telling Mike Cottmeyer this morning, a Scrum team in the right context will (and have) evolve into (many of) the Kanban practices.
Kanban explicitly calls out the rest of these principles. The Kanban itself becomes a framework for understanding the System and focusing on just enough information. The operations review is specifically in place to establish a betterment timeline and the solution after next.
Most of the Scrum teams dislike management and the other functional groups they interface with. The concept of pigs and chickens and the common perception that management causes obstacles for the team lead to conflict and distrust. Kanban focuses on how the development team can work better across the organization. From a purposes focus and people design standpoint, the Kanban perspective leads to greater trust and a shared focus on the solution-after-next.
Finally, many Scrum teams struggle with how to schedule defects and rework into their iterations. In the iteration world, QA gets the release after the end of the iteration for acceptance testing – this creates delay in feedback. This delay leads to all the relearning problems that we all have discussing. The enforced reduction in work in process and the single-unit flow reduces rework and shortens the time to feedback on defects and acceptance issues.
According to most reports, we still aren’t great a reliably delivering technology. No one has an answer yet that works for everyone. There is a lot of room for improvement and I believe we need to keep an open mind as we uncover better ways of developing software by doing it and helping others do it.
