It’s the Agile Practioners that are not ready for the Enterprise
I spent last week in Miami at the Lean & Kanban conference. There was discussion (and twitter chat) about why Lean was better than Agile for the enterprise and how Lean and Kanban are different than Agile. But many of the distinctions were debatable because you can find Agile practioners that have adapted and inspected to include many of the Lean Software practices, including Kanban. I personally don’t see Lean and Agile as competitors. Lean is about improving flow through a value stream and Agile is about writing better software. Lean can help Agile teams perform better and realize benefits from agility at the Enterprise level. What was notable at the Lean & Kanban conference was the level of sophistication and maturity of the practioners that attended. These were experienced professionals focused on improving their organizations (and clients) ability to rapidly and profitably deliver value to their customers.
Big Agile Practices Survey
I gained some insight into the discussion about the distinction between Lean and Agile as I reviewed the results of Jurgen Appelo’s survey http://www.noop.nl/2009/05/the-big-agile-practices-survey-report-part-1.html into Agile practices. Agile is considered risky or even a pejorative by many Enterprises. Many executives don’t trust it as a mature or viable approach. The lack of trust that corporations have in Agile teams is not a reflection of “management’s” ignorance. It is a reflection of software teams failing to reliably deliver the software organizations and customers need. I have decided that the problem is not with Agile itself, it is in the level of process immaturity followed by many Agile Practioners.
For example, here are the practices that are not considered Agile in the Survey. The percentage reflects the number of practitioner’s who selected the practice as an Agile practice. Less than 50% means that the majority of the community does not consider it an Agile practice.
- Configuration Management: 25.2%
- Issue Tracking: 27.4%
- Use Cases: 28.2%
- Risk Management: 29.5%
- Design by Contract: 32.2%
- Software Metrics: 32.5%
- Usage Scenarios: 37.9%
- Code Reviews: 42.3%
- Root Cause Analysis: 42.6%
- Move People Around: 42.5%
The ones that jumped off the page to me were Configuration Management, Issue Tracking, Risk Management, and Software Metrics. Building Enterprise or commercial software without caring deeply about these items is problematic. An Agile team, who is interested in continuous delivery of valuable software pretty much has to care about these items. You can’t inspect and adapt without metrics and an understanding of your issues. You can’t continue to deliver the next release without configuration management. And Agile is all about Risk Management.
In fact, you hear people who believe that Agile is a no documentation, no planning, no requirements, and no control free for all. Apparently, many of these are Agile practioners themselves. This is incorrect. Agile is a disciplined process that, when done correctly, helps companies improve their ability to reliably deliver valuable software to the business. So how does the immature implementation of Agile become so common? There are a lot of forces at work here. I think the lack of “maturity” in many agile teams is the result of two different forces – the reaction to bad process and the local optimization problem.
A Reaction to Bad Process
Agile, at least partially, arose as a reaction to bad process. I think the pendulum has swung too far away from responsible software practices in response to costly and painful implementations of traditional big ceremony around things like risk management, issue tracking, requirements management, and configuration management. As a reaction, some Agilists believe that just enough, just in time documentation means no documentation. It doesn’t. Or that responding to change means no planning and no requirements. Not accurate. I got a tweet today that “By definition, agile programming dispenses with process.” What? One of the things that Agile is supposed to do is to eliminate the pain of big ceremony for its own sake and embed the purposes of these processes into your daily work. But a failure to understand the principles behind the practices leads to a large community (according to the survey – a majority) who don’t build software responsibly. Agile is a high discipline, introspective, continuously improving approach focused on delivering customer value. A reaction to bad process is not an excuse for no process, it is a basis for figuring out other ways to achieve the right purpose.
Local Optimization
Local optimization is the second major force. Local optimization is where one group focuses on improving its ability to do its job at the cost of the organization. Development organizations, like most functional groups, operate within their sphere of control. They can fall into the trap of believing that the business revolves around them. In an effort to improve development performance, Agile teams implement practices they have read about or experienced without understanding how they fit into the overall value stream. Gaps arise (i.e. configuration management) when there is a lack of clarity on responsibility across the value stream. Agile teams also don’t recognize the additional strain on other parts of the organization when they implement new Agile practices. The only time you should subordinate one functional area for another one is to elevate or eliminate a constraint. The goal of the organization is to improve the organization’s ability to profitably deliver value to the customer, not to get better at software development.
Agile Practioners are Solving the Wrong Problem
When implemented maturely, Agile works to improve quality, fit, reliability, and responsiveness. The forces of reaction to bad process and local optimization result in Agile teams implementing not enough process or focusing on the wrong processes. The difference in (many) Agile practioners and Lean practioners is the problem they are solving. Immature Agile practitioners are focused solely on writing code better. This is important, but not a worthy goal in and of itself. Lean practioners (and mature Agile practioners) look at the overall value stream. They strive to identify the right process (and amount of process) while focusing on eliminating waste in the value stream. The problems with Agile are not a problem with the principles and practices of Agile. It is a problem with the perspective and purpose on the part of many Agile practioners.
Subscribe via RSS
Dennis,
Nicely done. I agree that Agile teams have not won the trust of upper management – but overall I’d say they were set up to fail. They were given the autonomy to fail, but not the support to succeed.
Quite often, agile in organizations are agile overlays of a rigidly defined external process.
Now, is this a still a problem with the practitioners? Usually, yes. Because they know the mechanics of agile, but don’t adequately understand the principles.
Where Lean helps with better integration of agile teams is that it provides a set of principles understandable to both sides that serve as a cultural translation layer between them. This helps undo a lot of the combative rhetoric that was unfortunately part of agile.
I, personally, think that in the maturation of software development processes, the rebellious adolescent phase of agile was necessary. Now, can we mature without losing the benefits of agile? I sure hope so.
Dennis,
Nice post! A contributor to the debate is that many people view Lean as a collection of tools, and not as a culture of continuous improvement. To use the tools of Lean is not the same as being Lean. I comment more on Lean, IT, and Agile here: http://itbusinessalignment.wordpress.com/2009/02/02/lean-it-and-agile/
Glenn,
I participated in your survey and agree with your personal judgment. I also agree that Lean is more than a collection of tools. I believe that most executives are understanding that implementing Lean results in/requires a change in the culture. This is consistent with the discussions at the Lean and Kanban conference in Miami that broad cultural change occurred as part of implementing Kanban. I have had the same experience with the Capabilities Analysis work I have done (which has a part of its roots in Value Stream Mapping). One executive, who we had found substantial savings for, said the most important result was that his employees were thinking about how they integrated into the business to create value instead of just how to make their job better.
Thanks for following me.
Ignoring for the moment the statistical confidence in the survey, the simple implication is that agile from the survey is about writing code, not managing the project.
This of course is not a problem, if someone IS managing the project. It becomes a problem if the agile development team thinks they’re managing the project.
Glen,
Great point. That’s the situation a lot of teams find themselves in. In Scrum, there is no concept of a project manager. The Scrum Master ensures the agile processes are followed and handles some of the PM responsibilities. The team itself is “self-managed” within certain constraints. These constraints are set by the Product Owner. Product Owners tend to think of themselves as responsible for marketing and prioritizing features/stories.
But there is a ton of work required between concept to delivery that isn’t about writing software. The PM items that include procurement, human resource management, most risk management, and coordination of most downstream activities, etc. are left to “self-organize”. Which means they just don’t get done very well.
Dennis,
Agree 100% with the ton of work needed. When I engage in the Agile conversation – I’ve learned to stay away actually – I use CMMI-DEV V1.2 as my framework. Of course it IS a software development framework.
I ask “what process areas are covered” in your agile method. Technical Solution has pretty coverage, but the other three process groups have no coverage.
This usually ends the conversation with “well that CMMI stuff is just heavy weight waterfall.”
I think much of the problems you cite stem from the fact that often it is individuals within R&D organizations who are driving the change to Agile, not the senior management. These individuals for good reason want to get away from waterfall methodologies, but fail to fully implement agile processes as they have no prior experience of agile and it takes time for them to realize and understand the potential benefits of the various practices. In addition, they are often self taught because the senior management support isn’t there and so the necessary trailing isn’t readily available to them. In short, the agile process takes time to mature within organizations, and I think this is why you see some of the surprising results in the survey.