Feeding the Software Development Pipeline

Unless a company is a software development service provider, its business is not about software development. Software likely enhances the businesses ability to create value for it’s customers. So, while Agile development helps teams get better at software development – the goal is to get better at creating value for it’s customers. The focus of Agile software development is upside down. The focus should be on the capabilities of the business. Aligning development with the needs of the business requires understanding  which business capabilities will benefit from improvement and prioritizing these needs based on business value.

After using a business capability approach to need identification and prioritization, the business needs must be appropriately scoped and communicated to development. Over the last month I have been writing about Making Agile Requirements Work. need to make sure the requests that go into the Software Development Pipeline are clear and appropriately defined. Following the six principles of Agile Requirements That Work help with this feeding of the Software Development Pipeline.

1. Articulate Purpose and Outcome
2. Establish a Shared Language
3. Provide a Stable View of the Business
4. Support Progressive Elaboration
5. Testable
6. Prioritized and Reflect Business Value and Risk

Gaining Business Benefit

The other part of the equation is to ensure the organization adopts and supports the improved software. Once the Software Development Pipeline has produced Improved Software, it still isn’t valuable until the Business Capability has achieved the intended performance improvement. Just as the elicitation and analysis of requirements has to match the cadence and needs of Software Development. Software Development has to match the cadence and needs of the organizations ability to adopt and support Improved Software. Part of the overall feedback to the organization includes ensuring the capability improvement is achieved.

Risk

At every step in this equation different risks may exist. identifying, prioritizing, and scoping feature requests have risk associated with a lack of understanding. Software Development has risk associated with the Ability to Execute and Verify the delivered software. Benefit realization has risks associated with achieving adoption and being able to sustain and support change in the organization. These risks should be explicitly managed and feedback mechanisms should be in place to recognize the realization of risks. Failing to understand and manage risks in an integrated fashion will also diminish the organization’s ability to maximize benefit from it’s investment.

The Real Agile Bargain

The real agile bargain is the maximizing of economic outcomes by increasing the speed and focus an organization can improve and respond to the market. The Agile Business Value Equation is solved at the Business Capability Level. The ability to rapidly develop high quality software moves Technology from a change obstacle to become an change enabler. This happens at the intersection of Feeding the Software Development Pipeline, Developing the Software, and Gaining Business Benefit. Solving any of these parts independent of each other does not result in optimizing business agility.

BABOK® is consistent with this. It defines a requirement as:

1. A condition or capability needed by a stakeholder to solve a problem or achieve an objective
2. A condition or capability that must be met or possessed by a solution or solution component to satisfy a contract, standard, specification, or other formally imposed documents.
3. A documented representation of a condition or capability as in (1) or (2).

The purpose of documenting requirements is not to document detailed steps that someone performs – it is to communicate the outcome needed to achieve an objective. One problem that arises is when requirements are expressed as a document of the existing process – typically through a review of existing process documentation or through observation of people doing the job. As I have discussed in previous posts, just documenting process can lead you into the How Trap and it can fail to generate a useful shared understanding. These problems arise when the focus is on how stakeholders are currently doing the work and not on understanding the challenges, problem or objective.  But, focusing on process is a problem in another way. It is an unstable lens.

Unstable

Modeling processes early in the elicitation process is not helpful to creating an appropriate understanding of the effort. Process reflects how people say they ought to do a task in perfect circumstances. The work is almost never consistently performed based on process documentation – or even how you observe the process being performed. Exceptions, reorganizations, changes in personnel within or adjacent to process, new tools and technologies, and innovation will change the process view. Even just asking five different people what the process is will likely produce five different results.

The process view can also propagate flawed understanding on the part of the people doing the work. You have likely heard the story about the recently married young women who was cooking her first Thanksgiving dinner in her own home. She had invited her family over the beautiful house she shared with her husband to show her parents how successful she had become and to demonstrate her gratitude to her parents. As she brought out the turkey for dinner her mother noticed the young women had cut the tail off of the turkey. The mother asked her daughter why she had cut the tail off the turkey. “Because that’s they way you always did it, Mom.” The mother smiled and replied, “That’s because the only pan we owned was too small for the turkey.”

A process based view hides assumption, complexity and is an unstable representation. The real requirement doesn’t change based on who you ask, who is doing the job, or the most recent reorganization. Basing requirements on unstable views doesn’t reflect the intent of the solution. This may result in an expression of the solution that doesn’t meet the needs of the business. As you observe a process or review process documentation ask why each step is important and what the intent of each step is. Consider what would happen if the steps were done in a different order or something was left out. Creating a stable, purpose driven view of the objective will provide valuable input to the development organization that will result in an improved solution.

Prioritize Requirements

Organize Requirements

Specify and Model Requirements

Define Assumptions and Constraints

Verify Requirements

Validate Requirements

Agile Requirements Analysis
All of the tasks in Requirements Analysis are important on Agile projects. The primary distinction in this Knowledge area is the support for the emergence of the solution over time. Particularly interesting is Define Assumptions and Constraints. While many Agile teams focus on the impediments to delivering code, they fail to manage the bigger risks and constraints associated with getting to business value.

Requirements Management and Communication: BABOK® Tasks

Prioritize Requirements

Organize Requirements

Specify and Model Requirements

Define Assumptions and Constraints

Verify Requirements

Validate Requirements

Agile Requirements Management and Communication

All of the tasks in Solution Verification and Validation are important on Agile projects. The primary distinctions in this Knowledge area are the support for the emergence of the solution over time. Particularly interesting are Assess Organizational Readiness and Transition Requirements as these are often overlooked in Agile projects (many software development projects). Remember, the product isn’t done-done until the customer is using it and achieving the benefits expected by the product owner.

So, What’s New?

Agile methods have emerged from a few simple concepts. Change happens and people trump process (Thanks to Ken Schwaber via Steve Adolph). In response to this, substantially all of the Agile methods describe a way of delivering software that support the following attributes:

• Progressive elaboration a little ahead of demand (for planning,  requirements,  architecture, testing,  etc)
• Incremental and iterative delivery (though not time-boxed in Kanban/Lean continuous flow)
• Ongoing learning through close customer engagement regarding product fit/business value that informs requirements and prioritization
• Ongoing learning regarding development/delivery approach and interactions that informs ongoing improvement efforts
• Self organization around the work (where the people performing the work decides how best to do their jobs within organizationally appropriate constraints)

In traditional software development, this identification and prioritization of what you are going to build falls in the role of the Business Analyst. However, traditional business analysis methods do not operate in harmony with the iterative and incremental cadence of Agile methods. For the most part, this is different than traditional plan driven projects.  Over the next week or so, I am going through the capabilities associated with Business Analysis and discuss the how Agile impacts how business analysis is performed.

The Guide to the Business Analysis Body Of Knowledge

I am going to build this discussion on work from The Business Analysis Body of Knowledge® (BABOK®) from the International Institute of Business Analysis. The IIBA® is the independent non-profit professional association serving the growing field of Business Analysis.  The IIBA has collected the current generally accepted practices within Business Analysis in The Business Analysis Body of Knowledge® (BABOK®).The BABOK® Guide describes Business Analysis areas of knowledge, their associated activities and the tasks and skills necessary to be effective in their execution.

The BABOK has six Knowledge Areas.

• Business Analysis Planning and Monitoring: Determine which business analysis activities are necessary to support an effort.
• Elicitation: Understand the underlying needs rather than stated or superficial desires.
• Enterprise Analysis: Understand what the enterprise is capable of performing.
• Solution Verification and Validation: Determine which solution best fits the business need – includes assessing the performance and effectiveness of the solution.
• Requirements Analysis: Organize, prioritize, and model requirements and risks.
• Requirements Management and Communication: Ensure knowledge gained through business analysis activities throughout the effort is shared among the team.

Within each Knowledge Area are several tasks. I will focus on how progressive elaboration, incremental delivery, close customer engagement, ongoing learning, and self organization impact who performs business analysis tasks and how business analysis tasks are performed.

Agile Business Analysis

Just to set my context. All of the Capabilities that are reflected in the BABOK happen on all projects – traditional or Agile. They may or may not be done by a specific role. They may or may not happen in a specific phase. They may or may not be done with any ceremony or intention. They do occur. Since the identification, prioritization, and scoping of requirements is so important to success, they need to be done well. And on an Agile project they need to be done so they enable – not constrain or impede – the delivery of value in projects leveraging progressive elaboration, incremental delivery, close customer engagement, ongoing learning, and self organization.

Enterprise Agility

Many companies are dependent on technology to be competitive. Either because they deliver technology as their product or because technology enables the processes they depend on. For those companies it is important to be able to operate faster than their competition.  This requires not only the ability to rapidly deliver technology – but the ability to adapt their organization.  This is the concept of operating within your competitors decision cycle. The model I use to demonstrate this concept is John Boyd’s O-O-D-A model.

This is Enterprise Agility, when the enterprise has learned to leverage technology and change management to develop an energized workforce frequently delivering value that meets the emerging needs of the customer.

Three Concepts for Achieving the Agile Enterprise

I am going to build on three concepts in showing how to build Enterprise Agility.

1. Theory of Constraints. You can’t get there all at once. It is necessary to decide where to focus. You have to develop the underlying capabilities from the bottom up. If you can’t produce quality product, the best ideas in the world will run into problems. The way of thinking about problems covered in the management system the Theory of Constraints provides the method for deciding where to focus. Identify the current constraint, get the most out of the capability that is the constraint, subordinate the system to the constraint, elevate the constraint, and then continue to pursue this cycle. This “Process of On-Going Improvement” is applied to the flow of value to the customer.

2. Creative Solution Finding. Once we understand where to change we need to understand what to change to. We have to apply a method to lead the overall adoption effort that supports aligning the capabilities with the overall goals of the organization. The stages of Creative Solution Finding are useful for identifying the situation specific approach to developing this road-map. There is no magic sauce for developing this road-map. By applying the seven principles, along with a knowledge of applicable best practices, a useful road-map that fits both the process and social context of the Enterprise can be developed.

3. The Learning Organization. Technology development efforts are knowledge based efforts. As such, they are performed through a social construct. To improve the performance of software development we have to not only improve the methods of development, we have to improve the performance of the social construct. First the individuals on the team must be willing to change. While the People Design principle in Creative Solution Finding will help prepare individuals for change, their personal fears and concerns must also be addressed. Once the constraint is identified and a road-map for improvement is developed, the learning organization concepts in the Fifth Discipline are relevant. The right individual competencies must be developed on the team to do the job (Personal Mastery and Mental Models).  The team has to be functioning well together (Shared Vision and Team Learning).  And the Team has to be functioning in a way that is aligned with the goal of the overall system (Systems Thinking).  The primary tool used to influence a social construct to create change and to improve performance is the conversation.

Discussing Big Agile

So the approach is to take the Big Agile capabilities, (1) provide insight to identify when a specific set of capabilities at a level of scaling are the current constraint, (2) leverage the solution finding model and best practices to define/refine the roadmap to move toward,  (3) and then identify the competencies to develop and conversations that are necessary to execute the change.  Then continue the cycle continuously.

This is the Big Agile improvement approach. It takes a model that I have been applying for two decades implicitly and evolving explicitly for the last decade. Clear direction combined with appropriate best practices and an effective adoption approach. All that’s left now is to build out the details of the capabilities, when they are a constraint, how to select appropriate best practices, and the conversations and change approach at each level of scaling for the model.

I have gone to dictionary.com to gain clarity on what it means to scale.  I pulled eight definitions to determine which one we intend.

Scale (skeyl)

1. one of the thin, flat, horny plates forming the covering of certain animals, as snakes, lizards, and pangolins
2. a balance or any of various other instruments or devices for weighing
3. a succession or progression of steps or degrees; graduated series:
4. a succession of tones ascending or descending according to fixed intervals
5. the proportion that a representation of an object bears to the object itself
6. Australian Informal. to ride on (public transportation) without paying the fare
7. Dentristy scal•ing. The removal of calculus and other deposits on the teeth by means of instruments
8. a certain relative or proportionate size or extent

We don’t mean what comes off fish, what we weigh things with, a way of measuring temperature, a musical construct, making a model of something, riding without paying the fare, or cleaning teeth.

So we must mean taking Agile to a proportionate size. That’s to point of the different orders of scaling. That Agile will be expressed differently at each order. But what are we actually taking to a proportionate size? Do we want to have an Enterprise wide stand-up with thousands of people? Do we want to have every job done by two people? What are we scaling?

Small team agile, as guided by the Agile Manifesto for Software Development, is a set of values and principles. Agile, as practiced in many development teams, is a set of management, leadership, and engineering practices resulting in an energized workforce frequently delivering software that meets the emerging needs of the customer. For other definitions of Agile, I went back to dictionary.com.

Agile (aj-ahyl)

1. quick and well-coordinated in movement
2. marked by an ability to think quickly

Well, we aren’t hoping that every person in the Enterprise learns to write code. It isn’t the specific practices that we want to see scale. It is quick and well coordinated movement we see in small team agile. It is the ability to think quickly and respond to emerging customer needs.

What do we mean by scaling agile to the enterprise? We are scaling the benefits of agile – the benefits of an energized workforce frequently delivering value that meets the emerging needs of the customer. We want to take this up from small development teams  so that the entire organization can be more adaptive and responsive.

It turns out that the capabilities (not the specific practices) that make the benefits of small team agile possible can be expressed at each order of scaling. And when layered together, the benefits of agile can be scaled to the enterprise level. Over the next few weeks, I am going to go through the management, leadership, and engineering capabilities of small team agile and discuss how the capabilities scale to deliver benefits at the different orders of scaling.

First off, The Agile Manifesto really applies to software development. Small Team Agile is about getting teams together to develop software in a way that meets certain criteria. Agile teams operate as a healthy team and they reliably deliver a continual stream of high-quality valuable software. Valuable means that it serves some purpose for the customer. There are engineering practices, management practices, and leadership practices that combine together to enable Agile teams to be successful. A key benefit of Agile teams is that you can produce the most valuable features first, and continue to invest until the economic value of the code has been realized. Agile teams are different then most traditional software development teams because of the focus on small incremental delivery of always releasable code and the focus on pushing decision rights down to (empowering) the team.

Scaling Agile

I believe there are four distinct levels of Agile adoption. This is not four maturity levels. It is distinct levels where the engineering, management, and leadership capabilities express themselves differently to support the expanding context. In most organizations the scaling will not be linear – it will be implemented across these levels of scaling based on the needs of the business.

Horizontal Scaling is the first level of scaling. This is the ability to get multiple teams to deliver a continual stream of high-quality valuable software. What happens when you begin to have the ability to spread Agile across teams is you begin to have the ability to strike a new bargain with the business. You can help product management interface with the business and customers in a new way where you can be responsive to individual customer requirements and changes in preferences in the market.

First Order Scaling is where multiple teams are coordinated across a common project or product. Once you have teams reliably performing, a new level of coordination arises. Your constraint is no longer in the pace of development. New opportunities to exploit speed and responsiveness arise across product development are possible. Taking advantage of these new opportunities requires the engineering, management, and leadership practices to be expressed in new ways. For example, Architecture becomes more important from a common perspective. Agile, this raises the ability to strike new bargains with the customer. Complex projects can be developed in a more responsive fashion.

Second Order Scaling is where multiple teams are coordinated across multiple projects and products. This is where you may begin to see economic benefits from reuse across products. The new bargain you strike with the business is now at the portfolio level. The business can be more flexible in investment into products and projects and then shift to other efforts when the economic benefit has been maximized.

Third Order Scaling (the Agile Enteprise) is where the Agile Enterprise exists. This is where the business leverages the ability to continually deliver high quality valuable products beyond the product development effort. New bargains can be struck in Customer Service, Sales and Marketing, Invoicing, and every customer facing part of the business. Faster learning from the market, customer preferences, and other environmental inputs is leveraged to rapidly produce products that profitably adapt to what we learn.  Other internal functions are also impacted. In fact, third order scaling often comes into play in even Small Team Agile. Once you can benefit from (or have to take into account) making changes to the way value is delivered to your customer beyond the product development organization, you need to think about Third Order Scaling.

Big Agile and Orders of Scaling

Scaling Agile has different meanings depending on the respective scale. From effective small teams, to reliable small team adoption, through products, portfolio’s to the organization -the focus is on achieving scale appropriate outcomes and purposes while maintaining focused value creation at the Agile team level. You can’t execute an Agile strategy from the top if you can’t first operate in a reliable agile fashion at the small team level.