I have been reading a lot about of the recent group discussions and papers regarding Kanban and Lean Software Development. One of the things I have noticed is that there is not a consistent application of the concepts regarding time. One of the reasons we establish a body of knowledge is to create a common language that helps us communicate based on a shared understanding. This inconsistency in the use of terms around time create a gap in understanding. In an effort to help the conversation here are the appropriate definitions for each type of time.

Durations

Duration is simply a measure of elapsed time.

Lead Time: The average time it takes for one unit to go through the entire process – from start to finish – including time waiting between sub-processes. Lead time can take different perspectives.

Development Lead Time is the time it takes for a unit to go through development from release from the backlog until it is done (which includes analysis, development, test, acceptance, deployment – but maybe not release to production). It is interesting to the development manager.

Order Lead Time is the time that elapses between when the customer places an order and when the customer receives what they ordered. In cases where requests are queued in a back log and releases are decoupled from development completion this can be far longer than the Development Lead Time. This is interesting to customer.

Order to Cash Lead Time is the time from when the customer places the order to the time cash is received by the business. While this is often longer than the Order Lead Time it can be shorter than the Order Lead Time when the customer prepays. This is interesting to the person paying the bills.

Process Lead Time: The time the unit is being worked on in a specific process before it is passed on to the next process. For example, within Development Lead Time this could be analysis, development, automated test, acceptance test or deployment time. This is interesting to the person performing the specific process or activity.

Processing Time: The time the unit is actually being worked on. It would be the sum of all the Process Lead Times. This is interesting to use as we try to improve our Lead Time.

Value Add Time: The Processing Time spent on a unit that transforms the unit in a way that the customer is willing to pay for. Not all Processing Time is Value Add Time – although non-Value Add Time can be necessary due to compliance requirements, to optimize performance of value add resources, and to minimize rework. This is interesting as it has been the subject of much of the lightweight process effort in Agile development. When taken to far it results in increased Rework Time and increased Process Lead Time.

Rework Time: The Processing Time that results from poor quality.  Failure Demand is the amount of resource that is consumed by Rework Time. This is interesting as it is expensive.

Queue Time: The time a unit sits between sub-processes waiting for someone to work on it. This is interesting to us as we try to improve our Lead Time. This is no cost to improving flow to reduce Queue Time but it reduces Lead Time.

Cadences

Cadence is a measure of time per unit.

Cycle Time: The average time between units coming out the end of the pipe. This is interesting because it helps us predict Lead Time.

Takt Time: How often units need to come out the end of the pipe to meet customer demand. This is interesting because it reflects the rate we need to produce to optimize investment against revenue.

You would expect Lead Time to be the sum of Processing Time, Queue Time, and Rework Time – and it is in a purely serial process flow. But in cases where work moves in parallel the sum of Processing Time, Queue Time and Rework Time will be greater than the Lead Time. Lead Time is interesting because it is what we use to make commitments.  Improving Lead Time can be done by reducing Queue Time, shortening Process Lead Time, eliminating Rework Time, minimizing non-Value Add Time, and performing processes in parallel.

The concept of Value Add Time is interesting and has had strong impact on Agile processes. At some point things like documentation, planning, testing, and adding additional features are not adding value in a way the customer is willing to pay for. Reducing these items to the bare minimum time that is necessary to meet the needs of the customer is helpful. Reducing them to the point they increase rework or reduce the productivity of the value added performers is not helpful.

The biggest inconsistency I have noticed is the improper use of Lead Time (a duration), Cycle Time (a cadence), and Takt Time (a cadence). In summary, Lead Time is how long it takes us to build and deliver it, Cycle Time is how often a new one is produced, and Takt Time is the rate of customer demand. In a stable system, Duration and Cadence are related by Little’s Law. The cadence of Cycle Time will equal WIP divided by the duration of Lead Time.

I believe that consistent use of these terms will help us have more meaningful conversations and continue to improve the ways that we deliver software to our customers. Part of my focus over the last decade has been on reducing Lead Time through reducing WIP, Queue Time, and non-Value Add Time, improving quality to reduce Rework Time, and improving architecture, automation, and shared understanding to reduce Processing Time. Having a shared understanding for each type of time leads to productive conversations around specific techniques and improves the teams ability to focus on the right types of improvements.

Something I find interesting is the push back I hear from agile developers that process discipline will inhibit their creativity. They say, “Software development is a creative activity. If you put process rigor around it you will inhibit our creativity.” I have heard others complain about applying Lean concepts to software development. “This isn’t manufacturing,” they say, “There is no place for standard work in what we do.”

Non Sequiturs in Software Development

These statements are examples of the non sequitur fallacy. A non sequitur fallacy is where the argument is false because the conclusion (rigor will inhibit creativity) does not follow from the initial premise (software development is a creative process). We see examples of non sequitur’s every day. We see them in advertising. For example,

“Buy this car and you will be more appealing to the opposite sex”

This isn’t a true statement. The car won’t make you more appealing to the opposite sex. We may want to be –  so we believe it. But this is a logical fallacy because the conclusion (you will be more appealing) actually has no cause and effect relationship with the initial premise (Buy this car).

In fact, it is because software development is a creative process and because it isn’t manufacturing there there are a whole set of processes that must exist to drive value. It is important to focus creativity where it adds value. It is also important to create an environment where creativity can be harnessed. You may not be able to create standard work around domain specific problem solving, but I contend that the higher the level of process discipline in the team, the more reliably developers will  deliver value. Further, I believe that the lack of discipline in the following process areas is the key contributor to poor performance – particularly in new agile teams.

Areas Where Process Discipline will Drive Value

To make my point, I am going to discuss four processes areas where process discipline to the point of standard work will improve the delivery of software. “Standard Work” is defined as a simple written description of the safest, highest quality, and most efficient way known to perform a particular process or task. It is agreed by the team to be the only acceptable way to do the process it describes. It is expected to be continually improved.

This is not an exhaustive list of the processes areas where discipline is required. These are process areas that are critical to teams, regardless of your flavor of agile: quality, learning and feedback, collaboration processes, and flow management.

Quality: The purpose of Quality processes is to create an environment where it is safe to pursue creativity in domain specific problem solving. Quality processes ensure the resulting product meets the needs of the customer and supports the goals of the development team. Effective Quality processes result in a net improvement in time to value delivery. Test Driven Development, Continuous Integration, Use of Patterns, Configuration Management, and Coding Standards are examples of Quality processes. Quality processes don’t hinder creativity. Just as in manufacturing they enhance the ability to leverage creativity to deliver value.

Feedback and Learning: The purpose of Feedback and Learning processes is to inform the team about how they are performing and to identify where constraints to their performance exist. Armed with this information, the team takes corrective action to avoid problems and overcome impediments. They also identify improvements that increase their ability to deliver value. Feedback and Learning processes include maintaining Burn-down Charts, Cumulative Flow Diagrams, and Kanban Boards and performing Retrospectives and Operations Reviews. Because software development isn’t manufacturing, constant feedback and the resulting learning is necessary to ensure the Agile team is improving (or at least maintaining) its ability to deliver value to the organization. Feedback and Learning processes also provide the ability for teams to creatively improve their performance.

Collaboration: The purpose of Collaboration processes is to ensure the team has a clear understanding of what they need to deliver and have a shared understanding of the dependencies within the team. Effective Collaboration processes will dramatically improve the teams ability to create value. Sprint Planning, Daily Stand Ups, and Story Reviews are examples of Collaboration processes. Clearly, adherence and process discipline are necessary to ensure a team if performing effectively. Strict adherence to Collaboration processes does not restrict creativity.

Flow Management: The purpose of Flow Management processes is to prioritize work for the team to optimize value and to throttle the amount of work that is active within the team to optimize productivity. Management also uses Flow Management processes to understand where the business stands on commitments to stakeholders and customers. Setting Milestones, Iteration Planning, Backlog Grooming and setting WIP policies are examples of Flow Management processes. As we learned from manufacturing, following these processes with discipline can dramatically improve the rate and amount of value the team is able to produce. None of these processes restrict creativity.

Process without Purpose is the Culprit

When pursued with the appropriate purpose, process does not inhibit creativity – it enables creativity. Many of the processes and techniques that have proven to be valuable in manufacturing have also demonstrated tremendous benefit in software development organizations. The “bad-taste-in-your-mouth” feeling that developers get when discussing process discipline and standard work is the result of process for the sake of process. Don’t let that feeling sway you from putting appropriate process rigor into place. Remember, “Software development is a creative process. If you put process rigor around it you will inhibit our creativity” is a logical non sequitur. Process discipline and standard work are not inhibitors to creativity, they are enablers.

I spent last week in Miami at the Lean & Kanban conference. There was discussion (and twitter chat) about why Lean was better than Agile for the enterprise and how Lean and Kanban are different than Agile. But many of the distinctions were debatable because you can find Agile practioners that have adapted and inspected to include many of the Lean Software practices, including Kanban. I personally don’t see Lean and Agile as competitors. Lean is about improving flow through a value stream and Agile is about writing better software. Lean can help Agile teams perform better and realize benefits from agility at the Enterprise level. What was notable at the Lean & Kanban conference was the level of sophistication and maturity of the practioners that attended. These were experienced professionals focused on improving their organizations (and clients) ability to rapidly and profitably deliver value to their customers.

Big Agile Practices Survey

I gained some insight into the discussion about the distinction between Lean and Agile as I reviewed the results of Jurgen Appelo’s survey http://www.noop.nl/2009/05/the-big-agile-practices-survey-report-part-1.html into Agile practices. Agile is considered risky or even a pejorative by many Enterprises. Many executives don’t trust it as a mature or viable approach. The lack of trust that corporations have in Agile teams is not a reflection of “management’s” ignorance. It is a reflection of software teams failing to reliably deliver the software organizations and customers need. I have decided that the problem is not with Agile itself, it is in the level of process immaturity followed by many Agile Practioners.

For example, here are the practices that are not considered Agile in the Survey. The percentage reflects the number of practitioner’s who selected the practice as an Agile practice. Less than 50% means that the majority of the community does not consider it an Agile practice.

• Configuration Management: 25.2%
• Issue Tracking: 27.4%
• Use Cases: 28.2%
• Risk Management: 29.5%
• Design by Contract: 32.2%
• Software Metrics: 32.5%
• Usage Scenarios: 37.9%
• Code Reviews: 42.3%
• Root Cause Analysis: 42.6%
• Move People Around: 42.5%

The ones that jumped off the page to me were Configuration Management, Issue Tracking, Risk Management, and Software Metrics. Building Enterprise or commercial software without caring deeply about these items is problematic. An Agile team, who is interested in continuous delivery of valuable software pretty much has to care about these items. You can’t inspect and adapt without metrics and an understanding of your issues. You can’t continue to deliver the next release without configuration management. And Agile is all about Risk Management.

In fact, you hear people who believe that Agile is a no documentation, no planning, no requirements, and no control free for all. Apparently, many of these are Agile practioners themselves.  This is incorrect. Agile is a disciplined process that, when done correctly, helps companies improve their ability to reliably deliver valuable software to the business. So how does the immature implementation of Agile become so common? There are a lot of forces at work here.  I think the lack of “maturity” in many agile teams is the result of two different forces – the reaction to bad process and the local optimization problem.

A Reaction to Bad Process

Agile, at least partially, arose as a reaction to bad process. I think the pendulum has swung too far away from responsible software practices in response to costly and painful implementations of traditional big ceremony around things like risk management, issue tracking, requirements management, and configuration management. As a reaction, some Agilists believe that just enough, just in time documentation means no documentation. It doesn’t. Or that responding to change means no planning and no requirements. Not accurate. I got a tweet today that “By definition, agile programming dispenses with process.” What? One of the things that Agile is supposed to do is to eliminate the pain of big ceremony for its own sake and embed the purposes of these processes into your daily work. But a failure to understand the principles behind the practices leads to a large community (according to the survey – a majority) who don’t build software responsibly. Agile is a high discipline, introspective, continuously improving approach focused on delivering customer value. A reaction to bad process is not an excuse for no process, it is a basis for figuring out other ways to achieve the right purpose.

Local Optimization

Local optimization is the second major force. Local optimization is where one group focuses on improving its ability to do its job at the cost of the organization. Development organizations, like most functional groups, operate within their sphere of control. They can fall into the trap of believing that the business revolves around them. In an effort to improve development performance, Agile teams implement practices they have read about or experienced without understanding how they fit into the overall value stream. Gaps arise (i.e. configuration management) when there is a lack of clarity on responsibility across the value stream. Agile teams also don’t recognize the additional strain on other parts of the organization when they implement new Agile practices.  The only time you should subordinate one functional area for another one is to elevate or eliminate a constraint.  The goal of the organization is to improve the organization’s ability to profitably deliver value to the customer, not to get better at software development.  

Agile Practioners are Solving the Wrong Problem

When implemented maturely, Agile works to improve quality, fit, reliability, and responsiveness. The forces of reaction to bad process and local optimization result in Agile teams implementing not enough process or focusing on the wrong processes. The difference in (many) Agile practioners and Lean practioners is the problem they are solving. Immature Agile practitioners are focused solely on writing code better. This is important, but not a worthy goal in and of itself. Lean practioners (and mature Agile practioners) look at the overall value stream. They strive to identify the right process (and amount of process) while focusing on eliminating waste in the value stream.  The problems with Agile are not a problem with the principles and practices of Agile. It is a problem with the perspective and purpose on the part of many Agile practioners.