During the past several months I have been involved in a number of conversations regarding Lean, Agile, Kanban, and Traditional methods of software development. Most experienced people believe that there are appropriate practices from each of these methods and that each organization has to come up with their best implementation. In the discussions the question continues to come up whether it is more important to teach principles or practices to teams adopting new methods.  I think this is a premature discussion. Without an understanding of purpose – the direction you are heading will not be clear.

Start with Purpose

Principles and practices are both about “How”. Principles are about the theory of “How” and practices are about the mechanics of “How”. “How” discussions that happen out of context often result in dogmatic conflict. The right place to start is to focus on “What” you are trying to do and “Why” you are doing it. We need to focus on the purpose of what we are doing in the context of the business strategy. Then drill into appropriate principles and suitable practices to implement. All too often, teams are focused on “How” without clarity of the “What” or the “Why”.

Sounds great, right? So how do you do this? I have had dramatic success in helping organization’s achieve dramatic improvements in performance using a purposed based approach. The approach based on focusing on “What” and “Why” is called Business Capability Analysis.

What is Business Capability Analysis?

Business Capability Analysis is based on rapidly developing a model of the business to identify where to improve and facilitate discovery of the best way to improve. The common model gets everyone involved focused on the purposes involved and results in a shared understanding of what needs to be accomplished to achieve the business strategy. This understanding is critical before you start applying practices and principles.

Business Capability: a particular capacity or ability that the organization relies on for a specific purpose or outcome.

• Business Capabilities are purpose focused. Their purpose is to produce value for the organization. What are we trying to do and why are we doing it? How does this capability help achieve the business’ strategy and operating objectives?
• Business Capabilities remain stable – even when the implementation changes. For example, almost every business has a “Pay-Employees” capability. Regardless of implementation, the regulatory requirements, employee expectation, interface with accounting, and many other factors don’t change. What you are trying to do and why you are doing it remains the same whether you automate it, outsource it, or are doing it manually.
• Business Capabilities provide a common business focused model for other attributes. Ownership, cost per, timing, and other interactions are a sample of the attributes that can be captured within a capability. Business Capabilities can be used to align the organizational, technology, process, and information of a business.
• Business Capability performance is dependent on the interactions with other capabilities.

Business Capability Example

Let’s use “Travel-To-Work” as an example. The purpose of “Travel-To-Work” is to get where you need to be so you can do your job every day. It is important that you are on time reliably. Based on your objectives, you may implement this capability differently. This choice will drive organization, process, and technology decisions. Here are a few examples of implementing this capability based on differing objectives.

So the capability is stable even when the implementation changes. While we can debate on appropriate implementations, an interesting aspect of this approach is that it frees us up to discuss alternatives to achieving the purpose of the capability. Remember, the purpose is not to travel to work. The purpose is to be where you need to be to do your job. Why do we need to physically travel to work? Another approach might be to provide access to the computer systems and telecommunications. By looking carefully at the purpose of the Capability in the context of our objectives, we can come up with an innovative new implementation.

Capability Analysis Applied to Documenting Requirements

Let’s look at document requirements through the Business Capability lens. This is an area of significant conflict among the various software development approaches.  Agile advocates will tell you that there is no way to build perfect documentation up front – that this is a collaborative effort and any more than rudimentary documentation will result in waste. BUFD advocates will tell you that to the extent the project outcomes are knowable up front, it is irresponsible to fail to produce an appropriate requirements document. It is irresponsible because you don’t understand where you are relative to the promise made to the business relative to cost, time, and scope - you have no way to measure performance.

So, one of the purposes of documenting requirements in software development is to communicate what we want developers to build. Another is to establish a baseline so we can make sure we are on track to meet a commitment to the business.  There are two distinct capabilities here, Communicate-Needs-to-Development and Evaluate-Project-Progress.

In many software development organizations a highly detailed document may not be the best way to accomplish either of these.  Based on the nature of your organization’s projects and the level of clarity of the technologies and customer needs, these two capabilities will have different cycle times, different owners, and will interact with different capabilities.

The capability here is not Document Requirements. Documenting requirements is a means to an end. There is no business value in getting better at documenting requirements for the sake of documenting requirements. Conflict arises when the focus is on the “How” instead of the “What” and “Why”. Implementing practices or principles without a focus on purpose results in waste, conflict, and constraints to understanding. Taking the time to understand the actual capabilities and focusing on achieving the purpose will pay big dividends.

I spent last week in Miami at the Lean & Kanban conference. There was discussion (and twitter chat) about why Lean was better than Agile for the enterprise and how Lean and Kanban are different than Agile. But many of the distinctions were debatable because you can find Agile practioners that have adapted and inspected to include many of the Lean Software practices, including Kanban. I personally don’t see Lean and Agile as competitors. Lean is about improving flow through a value stream and Agile is about writing better software. Lean can help Agile teams perform better and realize benefits from agility at the Enterprise level. What was notable at the Lean & Kanban conference was the level of sophistication and maturity of the practioners that attended. These were experienced professionals focused on improving their organizations (and clients) ability to rapidly and profitably deliver value to their customers.

Big Agile Practices Survey

I gained some insight into the discussion about the distinction between Lean and Agile as I reviewed the results of Jurgen Appelo’s survey http://www.noop.nl/2009/05/the-big-agile-practices-survey-report-part-1.html into Agile practices. Agile is considered risky or even a pejorative by many Enterprises. Many executives don’t trust it as a mature or viable approach. The lack of trust that corporations have in Agile teams is not a reflection of “management’s” ignorance. It is a reflection of software teams failing to reliably deliver the software organizations and customers need. I have decided that the problem is not with Agile itself, it is in the level of process immaturity followed by many Agile Practioners.

For example, here are the practices that are not considered Agile in the Survey. The percentage reflects the number of practitioner’s who selected the practice as an Agile practice. Less than 50% means that the majority of the community does not consider it an Agile practice.

• Configuration Management: 25.2%
• Issue Tracking: 27.4%
• Use Cases: 28.2%
• Risk Management: 29.5%
• Design by Contract: 32.2%
• Software Metrics: 32.5%
• Usage Scenarios: 37.9%
• Code Reviews: 42.3%
• Root Cause Analysis: 42.6%
• Move People Around: 42.5%

The ones that jumped off the page to me were Configuration Management, Issue Tracking, Risk Management, and Software Metrics. Building Enterprise or commercial software without caring deeply about these items is problematic. An Agile team, who is interested in continuous delivery of valuable software pretty much has to care about these items. You can’t inspect and adapt without metrics and an understanding of your issues. You can’t continue to deliver the next release without configuration management. And Agile is all about Risk Management.

In fact, you hear people who believe that Agile is a no documentation, no planning, no requirements, and no control free for all. Apparently, many of these are Agile practioners themselves.  This is incorrect. Agile is a disciplined process that, when done correctly, helps companies improve their ability to reliably deliver valuable software to the business. So how does the immature implementation of Agile become so common? There are a lot of forces at work here.  I think the lack of “maturity” in many agile teams is the result of two different forces – the reaction to bad process and the local optimization problem.

A Reaction to Bad Process

Agile, at least partially, arose as a reaction to bad process. I think the pendulum has swung too far away from responsible software practices in response to costly and painful implementations of traditional big ceremony around things like risk management, issue tracking, requirements management, and configuration management. As a reaction, some Agilists believe that just enough, just in time documentation means no documentation. It doesn’t. Or that responding to change means no planning and no requirements. Not accurate. I got a tweet today that “By definition, agile programming dispenses with process.” What? One of the things that Agile is supposed to do is to eliminate the pain of big ceremony for its own sake and embed the purposes of these processes into your daily work. But a failure to understand the principles behind the practices leads to a large community (according to the survey - a majority) who don’t build software responsibly. Agile is a high discipline, introspective, continuously improving approach focused on delivering customer value. A reaction to bad process is not an excuse for no process, it is a basis for figuring out other ways to achieve the right purpose.

Local Optimization

Local optimization is the second major force. Local optimization is where one group focuses on improving its ability to do its job at the cost of the organization. Development organizations, like most functional groups, operate within their sphere of control. They can fall into the trap of believing that the business revolves around them. In an effort to improve development performance, Agile teams implement practices they have read about or experienced without understanding how they fit into the overall value stream. Gaps arise (i.e. configuration management) when there is a lack of clarity on responsibility across the value stream. Agile teams also don’t recognize the additional strain on other parts of the organization when they implement new Agile practices.  The only time you should subordinate one functional area for another one is to elevate or eliminate a constraint.  The goal of the organization is to improve the organization’s ability to profitably deliver value to the customer, not to get better at software development.  

Agile Practioners are Solving the Wrong Problem

When implemented maturely, Agile works to improve quality, fit, reliability, and responsiveness. The forces of reaction to bad process and local optimization result in Agile teams implementing not enough process or focusing on the wrong processes. The difference in (many) Agile practioners and Lean practioners is the problem they are solving. Immature Agile practitioners are focused solely on writing code better. This is important, but not a worthy goal in and of itself. Lean practioners (and mature Agile practioners) look at the overall value stream. They strive to identify the right process (and amount of process) while focusing on eliminating waste in the value stream.  The problems with Agile are not a problem with the principles and practices of Agile. It is a problem with the perspective and purpose on the part of many Agile practioners.

Mike Cottmeyer has also been talking about scaling Agile. In his recent post on Second Order Agile Scaling he talks about how important it is to reduce the number of active projects in the portfolio. In other words, the goal is to focus on finishing projects as fast as possible, not on starting them faster or keeping your resources busy. When the organization has too many active projects at a time, it overloads the system and actually slows down delivery. You will see this when you have too many applications open on your computer. One application runs fast. Two probably still run well. But if you open ten or more applications, all of them slow down.

 In the last year I have seen a company with almost as many active projects as IT resources. Another company had the philosophy to start projects early so they wouldn’t get cut when the budgets were revisited. Not surprisingly both of these companies are late and over budget on almost every project. This is not a talent or process problem, it is a scheduling problem. When you overload the system, the resources start multi-tasking. My friend Steve defines multi-tasking as the intentional stoppage of ongoing work to transition focus to another task area. This is a bad plan. You want to avoid overloading the system with multi-tasking.

I have written about Multi-taking before. Several of my favorite bloggers have also discussed multi-tasking. Reforming Project Management, Focused Performance, and Clarke Ching often explain the dreadful impact of multi-tasking. Multi-tasking is bad for a number of reasons. I am drawing on my post at http://blogs.synaptus.com from March 2007 to discuss four of them here in the context of the current conversation.

One: Starting sooner doesn’t mean you will finish sooner.

Let’s say you have three people that want you to do something. To simplify this example, each thing takes three days to perform. You want to make everyone happy so you start working everything as soon as possible. You switch between them each day to continue to show progress on each thing. So your work looks like this.

Scenario 1: ABCABCABC

You deliver “A” on day 7. “B” on day 8. And “C” on day 9.  In trying to make everyone happy, you pushed A and B back and didn’t do C any favors. If you had worked on these things in sequence, your work would look like this.

Scenario 2: AAABBBCCC

You deliver “A” on day 3. “B” on day 6. And “C” is still delivered on day 9. Starting sooner didn’t get work done sooner.

Two: Sometimes work needs to be expedited.

It just happens. In Scenario 2 above, you could just drop the expedited work into the queue as the next thing to do. You don’t interrupt other work in progress, and you will finish it just as fast as possible. Remember, starting it as soon as possible isn’t what’s important. Finishing it as soon as possible is what’s important. Mixing expedited work into the middle of the flow will slow everyone down. Because it takes an incremental approach and working software is delivered at the end of each iteration, Agile development enables this much better than BUFD projects.

Three: Customer requirements change over time.

When you take longer to deliver work, you raise the risk that the customer requirements will change.  You want to have the shortest amount of time from when the work starts to when the work is delivered to the customer. This will reduce the risk that the customer requirements will change on work in progress. When customer requirements change on work in process, it creates rework. Rework is extremely expensive.

When you run in an Agile fashion, you will schedule epics (or big features or capabilities) into the pipeline well in advance. You have an Order of Magnitude estimate of time and cost, a general idea of what problem it addresses, and a perspective on the value of the epic (or big feature or capability) that you use for prioritization. You don’t do the detailed requirements and design until just before you drop it into the pipeline to be built. This shortened time reduces the risk that the customer will change the requirement.

Four: Task switching is expensive.

As this NY times article points out, we are not effective at getting back to productive work when we are multi-tasking. One of the tenants of lean is to eliminate waste. There are Eight Wastes defined in Lean. Multi-taking directly contributes to six of these wastes. We create Defects from forgetting what we were doing. We create Inventory from work piling up when we switch away. We create additional Processing from having to put away work and then find it again. We create Waiting (interruption of flow) and then the customer forgets what they asked for or their requirements change. We create Overproduction because we have more work started then the system can efficiently process. Finally, we create Non-Utilized Talent through the stress that is created for the developer who is always behind and having switch between tasks. Multi-tasking is a bad investment.

Following an Agile approach increases productivity because it specifically focuses on reducing these wastes. This is not accidental. When I took the Certified Scrum Master class from Jeff Sutherland he said that among other things they had intentionally drawn on proven concepts from Lean. 

Excessive multi-tasking derails productivity.

Our work lives are probably impacted by most of these issues - often in combination. The multiplying effects of multitasking are devastating to performance in organizations. You can have the right people, the best processes, and a solid strategy. Multi-tasking can derail the productivity of the group. Fixing this problem can double the productivity of a work group. In reality, you can’t eliminate multi-tasking completely. This is a complex problem to address but it is worth the investment. Taking an Agile approach to work, prioritizing completion of work over starting work, and intentionally coordinating work across functions with the intention of reducing multi-tasking are the first steps.

There is a lot of focus on Agile practices in software development today. This is because, when practiced successfully, Agile engineering, collaboration, and management practices improve the ability to produce valuable software reliably and continuously.  These new practices also create new possibilities that can improve the organizations ability to profitably deliver value to the customer.  However, if you plan to realize the benefits of Agile by focusing only inside the software development organization, you are likely to be disappointed in the outcome.

When implementing Agile, there is a lot of time and energy invested in improving software development teams. Businesses don’t make this investment because they want to get better at software development. They are trying to improve their capabilities from “Identify Customer Need” all the way through to “Fulfill Customer Need”. The Agile team is only one part of the business ecosystem.

Agile Impacts the White Space

Implementing Agile practices in the software development team impacts their interactions with the other functions in the business.  The frequency, content, and context of the interactions are different in Agile. As much as the Scrum and XP practices help improve performance within the development team, addressing what happens in the white space between functions is the key to implementing Agile in an organization.  In an existing organization of any scale, these white spaces can’t be hidden behind a Product Owner where the challenges self-organize themselves out of existence.

Agile Teams have to integrate with existing Product Management. Based on their understanding of the market and customer needs Product Management has to make promises on the timing and profitability of products to a CFO who reports those promises to a board and to the public. They still have to support the BUFD (Big Up Front Design) Teams. This work is important to the organization and the Agile Team has to interact in a way that allows Product Management to keep these promises.

Agile Teams have to integrate with existing Project Management. Project Management has to make promises on releases regarding cost and timing. They manage risks (external to the Agile team), track and report status, coordinate procurement, arrange for resources, and manage the performance of the project through the phase gates to the governance committee. This work is important to the organization and the Agile Team has to interact in a way that allows Project Management to be successful.

Agile teams have to coordinate with Sales and Marketing, Accounting, Operations, Human Resources, and Customer Support to make sure they are aligned with the features and timing of a product release.  These functional areas are key to the operation of the business and the delivery of value to the customer.  The Agile Team has to interact in a way that allows them to perform successfully.

Optimize the Whole Value Stream

In most sizeable organizations the development team is not on an island with no dependencies. They need to operate in a way that they can help the other functions of the organization be successful (and vice versa).  Implementing Agile with a myopic focus on the development team and a lack of understanding of the interactions within the organization can actually increase the cost to the organization of meeting the needs of the customer.

To gain benefit from Agile, you need to understand how the capabilities fit together within the organization to create value for the customer. Focus on scenarios where Agile practices can improve the flow of value to the customer and create a competitive advantage. Identify the conflicts in the white spaces that are in conflict with the processes and metrics that currently exist. The decisions you make about these conflicts will determine the level of benefit you can get from implementing Agile. If your functional metrics, processes and incentives do not align behavior in the White Spaces you won’t get the benefit.

Mike Cottmeyer over at Leading Agile is talking about scaling Agile across multiple teams. This is an important conversation and must be addressed to get benefit from Agile practices. But to truly realize the promise of Agile an organization must intentionally improve the interactions in the white spaces as well.

Ric Merrifield, led the development at Microsoft of what has become the Microsoft Business Architecture Service offering. He is also a coauthor of mine on “The Next Revolution in Productivity”, a June 2008 Harvard Business Review article focused on case studies that highlight needs of the organization and the opportunity to rethink business operating models before making major technology changes. Under Ric’s leadership, I was a significant contributor to the development of the Business Architecture offering and performed several of the projects that serve as case studies in the HBR article. We have spent a lot of time together over the last five years and he is a really smart guy and a friend of mine.

The concepts we have been leveraging revolve around using capabilities to help organizations clearly connect their business model to customer value. Typically, executives don’t have a good view of this except through their intuition. Organization charts are about the chain of command. P&L’s provide a historical view but a limited in what to do to improve performance. Process charts are too granular and change frequently. Technology architecture doesn’t help clarify how the business model connects to delivering customer value. Using the capability model based approach we can describe “what” a business does and “why” it is important without getting tied up in “how” they do it.  This approach overcomes the limitations of the current views most executives have.

This is important stuff to be talking about and I have been harassing Ric for several years about getting online because we need to have a broader discussion around the concept of capabilities or “hows”. Ric finally signed up for Twitter about three months ago, you can follow him at ricmerrifield. 

He has also written a book “ReThink: A Manifesto for Cutting Costs and Boosting Innovation” coming out in May (I am mentioned in it so ;-D). The book is endorsed by guys like Jim Champy, Robert Scoble, and David Anderson so you know it’s good. The book guides readers through a strategic re-think to efficiently set priorities, achieve goals, reduce costs, and unlock hidden value in today’s challenging business environment and set a course tailored for them. I will let you know 

And now, Ric has launched a blog at http://ricmerrifield.com/blog/. This is an important concept and it matters to business leaders, project managers and technology agilists.  We need to get involved in this conversation and help shape it and spread the work. Rick has his first two posts up.  Please go read them and add Ric to your blogroll. 

Last week I became an OPM3® Certified Consultant. After spending almost three years as the Deputy Project Manager with the volunteer team developing OPM3 Second Edition, I spent a week taking the OPM3 Certified Consultant course and passed the final exam. I have received a lot of questions about what that means and whether there is any value in it. There are about 100 OPM3 Certified Consultants in the world. I want to talk about OPM3, project management standards, getting organizationally mature, and what being an OPM3 Certified Consultant means. This might not sound exciting, but hang with me. There are very few organizations that won’t benefit from getting better at implementing strategy.

What is OPM3?

OPM3 is a standard that was first published in December of 2003 by the Project Management Institute. OPM3 stands for Organizational Project Management Maturity Model. Per PMI, “Organizational project management is the systematic management of projects, programs, and portfolios in alignment with the achievement of strategic goals. The concept of organizational project management is based on the idea that there is a correlation between an organization’s capabilities in Project Management, Program Management, and Portfolio Management, and the organizations effectiveness in implementing strategy.”

[There is some conflict in the terms process and capability. I will use the term capability consistently through my discussion. A capability describes “what you do and why you do it”. A process describes “how”. PMI doesn’t describe “how” in their standards even though they call them processes.]

To put this simply, organization’s have to implement changes in their structure, processes, management practices, and/or technology to implement a strategy. This is difficult but there are some capabilities that have been show to support them in consistently implementing these changes.  These capabilities can be categorized into effectively and efficiently executing projects, the coordination of multiple projects to optimize resource utilization and project performance, and deciding which projects are the best ones for the organization to invest in based on the organizations strategy. These are Project Management, Program Management, and Portfolio Management respectively. OPM3 uses the capabilities identified in PMI’s Project Management Body of Knowledge-4^th Edition, The Standard for Program Management-Second Edition, and The Standard for Portfolio Management-Second Edition. More on standards in a minute.

OPM3 wraps a continuous process improvement approach around these capabilities. For example, when implementing a project you probably want to manage the scope of the project. Within most organizations -most of the time, that means you have to gather requirements, define the scope of the project, and create a work plan when planning a project. There are a lot of different ways to do this. It may make sense on some projects to do this once at the start of the project and on other projects, you should do it iteratively and in progressively more detail. What organizations that are good at executing strategy do is they define how they are going to manage scope. Then they pay attention to how well this works continue to improve their method over time. This is organizationally mature.  OPM3 breaks the capabilities down into Standardize, Measure, Control, and Improve.

OPM3 also includes a set of Organizational Enablers. These are the capabilities that must exist in an organization to support the implementation and ongoing existence of Project Management capabilities. These include things like project management training, project management sponsorship, and other structural, cultural, technological and human resource practices.

You can pick up a copy of the OPM3 standard from PMI’s website for about $100. This standard describes the approach in detail, lists all the capabilities, and includes a nice Self Assessment tool in the appendix.

Project Management Standards

The Project Management Body of Knowledge (PMBoK) is a catalog of capabilities (called “processes” by PMI)  related to the management of projects. When the first OPM3 standard was published, PMI had not published standards for Portfolio Management or Program Management.  These first versions came out in 2006 and have been significantly updated with the simultaneous release of new Editions of the standards for Project, Program, Portfolio, and OPM3 in December of 2008.

The new standards provide a comprehensive catalogue of the capabilities that cover Project, Program, and Portfolio Management. These standards show the capabilities that will work in most organizations most of the time. They are based on years of research and the input of thousands of volunteers. They give examples of existing practices within the capabilities.  Despite what some critics claim, they don’t tell you “how” to implement those capabilities, just “what” you should be thinking about. They also don’t tell you to do all the capabilities whether they make sense for you or not, follow a high ceremony waterfall process, or make the purpose of the organization about project management.

Getting Mature

Maturity is an interesting concept. It has a lot of meanings. Mature can be a show you wouldn’t like your kids to watch, a bill of exchange that is due, or a ripe or fully aged piece of fruit. The definition of maturity I am using here is fully developed. What makes organizational project management maturity an interesting concept in business is that fully developed is different in almost every business. It can also change over time within an organization as the needs for project management change.  The key to organizational project management maturity is not to try to implement every capability with a robust set of process maturity. But to implement the capabilities that are right for helping a specific organization implement their specific strategy. This includes implementing an appropriate level of continuous improvement and only the necessary organizational enablers.

I have written about this focused approach to improving project management performance here and here. Invest when it improves the organizations ability to drive value. The goal is not to get better at project management, it is to get better at profitably delivering value to your customers. Project Management maturity efforts should be tied directly to business results.

 OPM3 Certified Consultant

An OPM3 Certified Consultant isn’t just someone who understands the OPM3 standard. Just to get into the course OPM3 Certified Consultant course you must demonstrate comprehensive knowledge of the OPM3 standard through completion of a SAM assessment or an OPM3 fundamentals course. An OPM3 Certified Consultant has demonstrated significant program or project management experience, since a PMP or PgMP is an eligibility requirement. An OPM3 Certified Consultant also has significant assessing and/or consulting experience.

Once in the course, the consultant is trained on the OPM3 assessment methodology and tool set, and the improvement methodology and tool set. This is during an intense four day training course. The consultant must successfully complete an exam.  The handbook that walks you through this process can be found at the PMI website.

The Value of OPM3 Certified Consultants

Most businesses can benefit from getting better at executing their strategy. Improving Organizational Project Management Maturity is a method of improving the organization ability to execute their strategy. OPM3 provides a systematic approach to identify and implement Organizational Project Management improvements appropriate to each business. Project Management Maturity is a tricky thing – it requires insight and experience to focus improvements in project management so it improves the organizations ability to deliver value to its customers. An OPM3 Certified Consultant has an assessment and improvement methodology, a robust tool set, and has demonstrated the experience with identifying and implementing improvements based on PMI’s standards.

For me, the value of the certification includes the assessment and improvement methodology training I received, the time I have spent with other experienced Organization Project leaders, access to the assessment and improvement tools provided by PMI, and some branding. I can combine this with my experience in software development and IT Operations excellence and cost optimization to rapidly deliver value to organizations. For organizations looking for a way to rapidly improve the performance of thier projects, an OPM3 certified consultant with proven experience and expertise in your domain is a good option. Guided by a proven systematic approach, they can perform an assessment and create an improvement roadmap and tie the improvements to your business results. 

I have been having a lot of discussions lately with Mike Cottmeyer over at www.leadingagile.com about scaling Agile to the Enterprise. Today he has a great post, Teams are the Building Blocks of Agile. Mike is exactly right. In fact, teams are the building blocks of organizations. And promises are the glue between the teams. In realms of uncertainty such as knowledge work and software development, Donald Sull would say to focus on Promises over Process.

Managing the promises in an organization is called Promise-based Management. The primary idea in Promise-based management is that work moves through organizations based on a network of promises. And that the better the promising is, the faster the work will move. The whole business is a network of “if you can do this”, then “I can do this” promises. When this network works well, the business improves its ability to profitably create value for the customer. We can improve the performance of the network by improving the reliability of the promises made between teams in the organization.

If you read Mike’s post from a certain perspective, it sounds like Mike is saying the purpose of the organization is to make life easier for the developer’s. That isn’t what he is saying, though.  He is saying that for an Agile team to keep its promise, you have to set the Agile team up to be successful by providing the appropriate context and coordination.

The promise the agile development team makes is to rapidly produce high quality software for use by the business. This is probably so the business can keep the promises it makes to its customers and stakeholders. The Agile development team is willing and able to make these promises when the <Product Owner>Product Manager, Project Manager, Designer, Subject Matter Expert, Business Stakeholder, Business Analyst</Product Owner> is able to set the table for the team by getting the proper context and coordination in place. Mike defines this part of the promise on his blog. This context and coordination effort is the “If you can do this” part of the promise that the Product Owner and Agile development team make.

Once we set the Agile development team up for success, they will be able to much more rapidly produce high quality software for use by the business. The new capability to keep this promise opens up a whole new set of promises we can make to the customer. Exploiting this new capability is not just about software development, it requires us understand impact on the other processes, promises and teams in the organization. Scaling Agile to the organization isn’t about software development. It is about fundamentally changing the promises we can make to our customers.

Check out “Promise-based Management: The Essence of Execution” in Harvard Business Review. Glen Alleman talks about the HBR article. Hal Macomber’s site is a rich resource practical application of promises and promising and making work ready. Clarke Ching’s site is an excellent source of the application of promised based management to software development.

There is a lot of interesting conversation going on within the project management, product management, and agile software development communities. There is contention between (and within) the different communities regarding the value and benefit from the other domains. Sometimes one community is vying to prove their domain is superior to the others. Other times, a community is casting blame for limitations in their ability to perform to another domain. Much of the conversation is based on a limited understanding of what the other domains actually do. They also fall into the trap of generalizing a specific unpleasant experience to encompass the intent of the other community.

I believe this debate is counterproductive. At the end of the day, everyone’s interest should be to improve their organization’s ability to profitably create value for the customer. Each of these areas has something critical to contribute to this goal – but the benefits are only realized when the communities work together. I want to define a few terms, identify the overlap, and then suggest why this discussion is not just an academic exercise, but it an important part of the maturing of each domain.

First, some definitions:

Project Management

Let’s start with Project Management. The Project Management Institute has the following definitions of project, project management and project manager.

Project: A temporary endeavor undertaken to create a uique product, service or result.

Project Management: The application of knowledge, skills, tools, and techniques to project activities to meet the project requirements.

Project Manager: Person assigned by the performing organization to achieve the project objectives.

Project Management is not meant to be an obstacle to getting work done. In fact, the project manager is responsible for ensuring the project is successful.

Product Management

The Product Development and Management Association define product, product management and product manager.

Product: A term used to describe all goods, services, and knowledge sold. Products are bundles of attributes (features, functions, benefits, and uses) and can be either tangible, as in the case of physical goods, or intangible, as in the case of those associated with service benefits, or can be a combination of the two.

Product Management: Ensuring over time that a product or service profitably meets the needs of customers by continually monitoring and modifying the elements of the marketing mix, including: the product and its features, the communications strategy, distribution channels and price.

Product Manager: The person assigned responsibility for overseeing all of the various activities that concern a particular product.

So the Product has an ongoing aspect to it that the Project doesn’t. The Product Manager decides what needs to be built and also how the organization will sell it and support it.

A product has a life that lasts beyond any individual project.  The ongoing thing over at PMI is called a Program. Here is the PMI definition for Program and Program Management. Interestingly, PMI doesn’t define Program Manager in the Program Management standard.

Program: A group of related projects managed in a coordinated way to obtain benefits and control not available from managing them individually, Programs may include elements of related work outside of the scope of the discrete projects in the program.

Program Management: The centralized coordinated management of a program to achieve the program’s strategic objectives.

Project Management Isn’t Product Management

So project management isn’t product management. Neither is Program Management, However, there are projects to execute in the modifying of the product, so project management processes matter in product management.  Here is a Venn diagram that describes the relationship.

If you aren’t good at making changes to your product or the capabilities in your organization that relate to your product, you won’t be a successful company. So project management isn’t product management, but there is overlap in practices and both are important to your organization. Great ideas with no execution aren’t valuable.

Agile Software Development

Finally, let’s go with the Agile Manifesto as a definition of Agile.

Manifest for Agile Software Development

• Individual and Interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

Despite the perspective of many in the project management and product management communities, Agile is also not intended to be no planning, no documentation, irresponsible software development. According to http://agilemanifesto.org/history.html :

 “The Agile movement is not anti-methodology; in fact, many of us want to restore credibility to the word methodology. We want to restore a balance. We embrace modeling, but not in order to file some diagram in a dusty corporate repository. We embrace documentation, but not hundreds of pages of never-maintained and rarely-used tomes. We plan, but recognize the limits of planning in a turbulent environment.”

Agile is not Product Management

Agile is specifically about software development. In fact, the Agile Manifesto is short for the proper title of Manifesto for Agile Software Development. Agile is an approach to software development that includes practices and principles that can result in dramatically faster and higher quality (therefore less expensive) development of software. But the product is not just software. If the right features, communications strategy, distribution channels and price are established being faster, cheaper, and better at software development doesn’t matter. Product Management is critical to getting value from Agile Software Development.

Agile is not Project Management

 In very few cases is the product that is being sold to an organization’s customers just software. We’ve discussed the relationship of Project and Product Management above.  You need to be good at Project Managing all aspects of change to the product and the capabilities in the organization that support and are affected in any way by the sale, delivery, billing, and support of the product to the customer. Also, Agile isn’t concerned with Procurement – but procurement is important to the profitability of the company. And while iterations are a Risk Mitigation tool, Agile does not have a specific Risk Mitigation aspect to it.

Why this discussion matters

The following Venn diagram depicts the interdependency between these three domains.  Value is not delivered by any one of these domains. It is through an effective coordination between these domains that value is delivered to the customer. Value is identified, shaped, and then communicated to the customer by Product Management. Much value is created by the Agile development team itself. Value is created through aligning the other capabilities within the organization and often actually delivering the product to the customer through Project Management.

Let me quote Jim Highsmith from http://agilemanifesto.org/history.html again.

“In order to succeed in the new economy, to move aggressively into the era of e-business, e-commerce, and the web, companies have to rid themselves of their Dilbert manifestations of make-work and arcane policies. “

Most of the organization’s that I have been involved with that are failing to deliver software to the business or the business’ customers are failing to receive value because of deficiencies in one or all of the domains in this discussion. Often, the deficiencies are the result of conflict and lack of coordination with another domain. Becoming great at one area, while remaining poor in another, is not value to the customer, the business, or ultimately to the members of the organization. 

The discussion we need to be having is not about how each domain contributes to problems for the other domain.  This is an interdependent problem, not one that has its roots in any one domain. Bunkering down, defending incompatible practices, focusing on local optimization at the expense of the business is not going to work. We need to work together to answer this question. How do we align the product management and project management practices with the rapid, incremental delivery of working software to optimize benefits for our customers and profitability for our businesses?

I come from a big project, high ceromony background. I learned Software Engineering as contractor at IBM in the 80’s, was a Marine, and believe in the PMBoK (Deputy PM on OPM3® Second Edition). Over the last 10 years I have studied and learned Agile from books and through mentoring relationships. I have applied what I understood and have experienced improved performance. But I still thought of Agile/Scrum as a less than responsible way to do work.  I have heard a lot of push back on Agile and Scrum in blogs and on Twitter lately. “Scrum doesn’t work”, they are saying. “Scrum makes the devleopers life easier but hurts the customer”, is another one. “The lack of overall budget/timeline is a killer (only look 1 sprint forward, really)”, is a charge hurled this week. 

Last week, I went to the source and completed Jeff Sutherland’s Certified Scrum Master course. During the training with Jeff Sutherland last week, he presented a ScrumBut test. We are doing Scrum but… means you aren’t doing Scrum. Not doing Scrum means you likely won’t get the benefits other teams that are doing Scrum have claimed. Scrum, when done well, is a responsible way to develop software and if positioned maturly is a benefit to management. With permission from Jeff, here is the ScrumBut test. Take the test, add up your scores and divide the result by 9. 

Question 1 - Iterations

- No Iterations - 0
- Iterations > 6 weeks - 1
- Variable Length < 6 weeks - 2
- Fixed iteration length 6 weeks - 3
- Fixed iteration length 5 weeks - 4
- Fixed iteration length 4 weeks or less - 10

Question 2 - Testing within the Sprint

- No dedicated QA - 0
- Unit tested - 1
- Feature tested - 5
- Feature tested as soon as completed - 7 
- Software passes acceptance testing - 8
- Software is deployed - 10

Question 3 - Agile Specification

-No requirements - 0
- Big requirements documents - 1
- Poor user stories - 4
- Good requirements - 5
- Good user stories - 7
- Just enough, just in time specifications - 8
- Good user stories tied to specification as needed - 10

Question 4 - Product Owner

- No Product Owner - 0
- Product Owner who doesn’t understand Scrum - 1
- Product Owner who disrupts team - 2
- Product Owner not involved in team - 2
- Product Owner with clear product backlog estimated by team before Sprint Planning meeting (READY) - 5
- Product Owner with a release roadmap with data based on team velocity - 8
- Product Owner who motivates the team - 10

Question 5 - Product Backlog

- No Product Backlog - 0
- Multiple Product Backlogs - 1
- Single Product Backlog - 3
- Product Backlog clearly specified and prioritized by ROI (business value) before Sprint Planning (READY) - 5
- Product Owner has release plan based on Produce Backlog - 7
- Product Owner can measure ROI (business value) based on real revenue, cost per story point, or other metrics - 10

Question 6 - Estimates

- Product Backlog not estimated - 0
- Estimates not produced by team - 1
- Estimates not produced by planning poker - 5
- Estimates produced by planning poker by team - 8
- Estimate error < 10% - 10

Question 7 - Burndown Chart

- No burndown chart - 0
- Burndown chart no updated by team - 1
- Burndown chart in hours/days not accounting for work in progress (partial tasks burn down) - 2
- Burndown chart only burns down when task is done (TrackDone pattern) - 4
- Burndown only burns down when story is done - 5
- Add 3 points of team knows velocity
- Add two points if Product Owner release plan based on known velocity

 Question 8 - Team Disruption

- Manager or Project Leader disrupts team - 0
- Product Owner disrupts team - 1
- Managers, Project Leaders or Team Leaders telling people what to do - 3
- Have Project Leader and Scrum roles - 5
- No one disrupting team, only Scrum roles - 10

Question 9 - Team

- Tasks assigned to individual during Sprint Planning - 0
- Team members do not have any overlap in their area of expertise - 0
- No emergent leadership - one or more team members designated as a directive authority - 1
- Team does not have the necessary competency - 2
- Team commits collectively to Sprint goal and backlog - 7
- Team members collectively fight impediments during the sprint - 9
- Team is in hyperproductive state - 10 

My best recent project scored a 6.7 on this test and we didn’t call it Scrum - we just considered it responsible management of an analytics projecyt. We did perform really well in a tough situation though. If you score less than a 6 on average, you probably aren’t doing Scrum. That may be good or bad - but if you aren’t doing Scrum, don’t call it Scrum. If you aren’t performing at a hyperproductive level, look at the areas where you didn’t score as well. It is likely they will highlight the next most important impediment for you to focus on removing. Fix that and move forward.

This is a slightly updated presentation of a talk I gave at Atlanta’s Project Developer Days a few years ago. After attending a two day training session with Jeff Sutherland, I believe that this concept is relevant to the Agile and SCRUM Project Management conversation.  From my experience, improving the performance of the conversations on your projects greatly improves the performance of the project team. When we say we want to get better at communications, I believe we want to improve the quality and effectiveness of the project conversations. The project managers job in leading projects requires us to be good at recognizing what conversations need to take place, and facilitating them to a successful conclusion.