We have proposed that Agile Requirements must be built at a purpose and outcome level, must leverage shared language that communicates context and intent, and must be based on a stable view of the domain. But even with these in place requirements can still fail to support the way people solve problems and the way people learn. When we try to be to prescriptive and communicate too much detail initially it will reduce effectiveness – not improve agility.  Agile requirements must support progressive elaboration.

Don’t Support Progressive Elaboration

One of the faulty assumptions made in Requirements Elicitation is that we believe we can know up front exactly what we want. It is also faulty to believe that we should document requirements in too granular detail. In fact, neither of these is accurate. Requirements should support the concept of progressive elaboration. That is, they should be documented so that more specific detail unfolds during the course of the project. This addresses the flaw in both of assumptions. First, documenting what we know at a high level of detail allows us to establish a consistent context that we can communicate in more detail the more we learn. Getting to too much detail early can result in rework and inconsistency in communication.

Agile teams may break requirements into Themes, Epics, Stories, and then Tasks. This may be hierarchical, follow a story-map, or be connected in a mind-map.  Themes are be defined early on and directly connected to business value propositions. These are then progressively elaborated into Epics and Stories as needed to feed the team. As Stories move into a ready state for development they can be further broken down into tasks by the development team. This progressive elaboration allows the team to apply what it learns about the best way to deliver the project as the project unfolds. Critically, it also supports the way that people learn about the problem. Too much detail early on can be overwhelming and result in a disconnected or inconsistent understanding of the effort. Not enough detail later on results in a lack of shared understanding of the problem. Progressive elaboration supports effective communication, collaboration in solution finding, and the application of what the team learns as the project unfolds.

The Gilb’s have started a series of posts against Agile. The first on is Wrong Focus.

“It is not about working software. It is not about your preferred working process. It is not about user stories. It is not only about your customer or user. It is all about delivering, to your set of Stakeholders, value improvements to them, to what they care about, what makes their world better, within agreeable, minimum or pre-determined costs. If that is not the main focus, if that is not clear to everyone on the team, you will eventually loose! If your methods are not focusing on delivering that value, as todays most popular Agile methods are not, they will fail to deliver, they will become last years fad.”

They claim Agile software development is flawed because it focuses on improving the process and environment of the team developing software. These improvements result in improved customer collaboration that ensures the team better understands what the customer wants. From my experience, the combination of improved environment, appropriately sized process, and customer collaboration results in mature agile teams doing a great job of delivering the projects and products. But I understand their point. There is a difference between understanding the customer and focusing on delivering customer value.

Wrong Focus

Whether the software development organization is building software for external clients or to enable the organization’s business processes, there are always more things that can validly be improved than there are resources to do the work. Not all of the improvements will result in the same return of business value to the overall organization. Some benefits are more valuable to the organization than others. If you don’t pick the one that is most valuable to the business it is not the right investment for the business. In any of these improvements, there is more improving that is valid than is necessary to achieve the benefit. If the team works on improvements that aren’t the focus of the effort, “while they are there”,  they are not focusing on business value. This delays delivery of the effort and is not the right investment for the business.

Nothing in the Agile Manifesto, nor most of the Agile methods, give the tools to pick the right improvements to focus on. They also create an environment where it is extremely likely to work on more then is the minimum necessary to achieve the targeted business benefit. In many organizations, close customer collaboration can lead to “gold-plating” of features. If you offer the customer whatever they want – they will ask for it.

Wrong Focus in Practice

I have a customer right now that is going to build some software to support their field employees. The business goal is to achieve a significant cost reduction in the field through improving quality and efficiency in the field. The developers are extremely interested in developing mobile applications, distributed collaboration, GPS interfaces, image capture and markup, mapping, barcode, and training training. They have started a new business to build this product with the hope of selling it outside the business in the future. The challenge here is how do they clearly identify those improvements in the field and in dispatching that will deliver the significant reduction in field expense. If they go and and talk to the dispatchers they might find 30 things that could be done better. All of these improvements would result in an improvement in work conditions or performance. But they may not be aligned with the current business focus. The same will happen in the field.

Just because it will improve satisfaction or make someone’s job easier doesn’t mean it will help with the current business focus. How do they align absolutely the minimum amount of technology to deliver on these improvements? Especially when their interests are in this bigger, more robust solution. Especially when this limiting of features is not in line with their future business objectives. Nothing in Agile addresses how to deal with this conflict.

Where’s the Answer

I address this in Feeding the Agile Beast. This is the domain of Business Analysis. As the Gilb’s point out, most Agile methods abstract this from the team. They focus on improving their processes and collaboration. In Scrum, Business Analysis is abstracted behind the Product Owner. In XP, it is abstracted behind the Onsite Customer. I don’t agree with the implication from the Gilb’s that this abstraction is a deficiency in Agile. It is just outside the scope. I also don’t agree with Agile teams that close collaboration with the customer is sufficient to ensure business value is achieved. This is not an OR conversation. This needs to be handled as an AND conversation. This is the next thing we need to learn – how to keep an Agile team laser focused on business value AND maintain the benefits of understanding the customer that comes with Agile software development.

Dictionary.com defines a requirement as “that which is required; a thing demanded or obligatory”.

BABOK® is consistent with this. It defines a requirement as:

1. A condition or capability needed by a stakeholder to solve a problem or achieve an objective
2. A condition or capability that must be met or possessed by a solution or solution component to satisfy a contract, standard, specification, or other formally imposed documents.
3. A documented representation of a condition or capability as in (1) or (2).

The purpose of documenting requirements is not to document detailed steps that someone performs – it is to communicate the outcome needed to achieve an objective. One problem that arises is when requirements are expressed as a document of the existing process – typically through a review of existing process documentation or through observation of people doing the job. As I have discussed in previous posts, just documenting process can lead you into the How Trap and it can fail to generate a useful shared understanding. These problems arise when the focus is on how stakeholders are currently doing the work and not on understanding the challenges, problem or objective.  But, focusing on process is a problem in another way. It is an unstable lens.

Unstable

Modeling processes early in the elicitation process is not helpful to creating an appropriate understanding of the effort. Process reflects how people say they ought to do a task in perfect circumstances. The work is almost never consistently performed based on process documentation – or even how you observe the process being performed. Exceptions, reorganizations, changes in personnel within or adjacent to process, new tools and technologies, and innovation will change the process view. Even just asking five different people what the process is will likely produce five different results.

The process view can also propagate flawed understanding on the part of the people doing the work. You have likely heard the story about the recently married young women who was cooking her first Thanksgiving dinner in her own home. She had invited her family over the beautiful house she shared with her husband to show her parents how successful she had become and to demonstrate her gratitude to her parents. As she brought out the turkey for dinner her mother noticed the young women had cut the tail off of the turkey. The mother asked her daughter why she had cut the tail off the turkey. “Because that’s they way you always did it, Mom.” The mother smiled and replied, “That’s because the only pan we owned was too small for the turkey.”

A process based view hides assumption, complexity and is an unstable representation. The real requirement doesn’t change based on who you ask, who is doing the job, or the most recent reorganization. Basing requirements on unstable views doesn’t reflect the intent of the solution. This may result in an expression of the solution that doesn’t meet the needs of the business. As you observe a process or review process documentation ask why each step is important and what the intent of each step is. Consider what would happen if the steps were done in a different order or something was left out. Creating a stable, purpose driven view of the objective will provide valuable input to the development organization that will result in an improved solution.

The problem with many current methods of Elicit Requirements on software development projects is that they don’t add economic value to the solution effort. Many projects develop unrealistically detailed requirements in large batches up front. Due to the impossibility of accurately defining exactly what the effort requires they aren’t completely useful. In addition to resulting in churn and rework they require a high level of effort to maintain. In response to problems with Big Up-front Elicitation efforts some projects have resorted to “too little” requirement definition up front. This also results in expensive churn and a lack of project focus that delays the time to value. In both cases the transaction costs associated with producing and maintaining the documents offset any value they might bring.

The How Trap

The How Trap is a very human condition. If you see someone at the fax machine and ask them what they are doing, they will say “Sending a Fax”. That isn’t what they are doing though. That is “how” they are doing something – not what they are doing. The How Trap is very common. Ric Merrifield, in his book Re-Th!nk: A Business Manifesto for Cost Cutting and Innovation talks about Dick Fosbury and the Fosbury Flop. Dick Fosbury realized the How of the high jump was to go as high as possible and abandoned trying to perfect any of the existing techniques that involved jumping over the bar so you could land on your feet. His focus on the What of jumping as high as possible led him to jump over it backwards and land on his back. His gold medal in the 1968 Summer Olympics pointed out the wisdom of his approach. When requirements are elicited on a How-Trap basis assumptions are made that limit options that should be considered to accomplish the goal of the effort.

To avoid the how trap elicit requirements based on outcomes and purposes. Avoid getting trapped in How specific language. You can be pretty specific about the purpose or outcome without falling into non-productive detail. For example, the outcomes and purposes associated with Pay Employees doesn’t change much regardless of the implementation. People need to be paid a specific amount, there are specific tax requirements, and there are clear governance and compliance needs. These don’t change whether Pay Employees is outsourced, done manually, or automated internally.

Remember Fosbury’s goal was not to jump higher it was to clear the bar at its highest point. Sending a fax is never the requirement. It is something like Communicate Status or Confirm Order. This is a subtle distinction but frees up the team to apply their creativity to the How and will lead to significantly different outcomes.

Today I am discussing the tasks from the BABOK associated with Requirements Management and Communication. These are the tasks Ensure knowledge gained through business analysis activities throughout the effort is shared among the team. There are six tasks associated with Requirements Management and Communication. I show the BABOK® purpose. Then I discuss the impact on who and how in projects leveraging progressive elaboration, incremental delivery, close customer engagement, ongoing learning, and self organization.

Requirements Management and Communication: BABOK® Tasks

Prioritize Requirements

Organize Requirements

Specify and Model Requirements

Define Assumptions and Constraints

Verify Requirements

Validate Requirements

Agile Requirements Management and Communication

All of the tasks in Solution Verification and Validation are important on Agile projects. The primary distinctions in this Knowledge area are the support for the emergence of the solution over time. Particularly interesting are Assess Organizational Readiness and Transition Requirements as these are often overlooked in Agile projects (many software development projects). Remember, the product isn’t done-done until the customer is using it and achieving the benefits expected by the product owner.

Today I am discussing the tasks from the BABOK associated with Enterprise Analysis. These are the tasks to understand what the enterprise is capable of performing. There are five tasks associated with Enterprise Analysis. Again, I show the BABOK® purpose. Then I discuss the impact on who and how in projects leveraging progressive elaboration, incremental delivery, close customer engagement, ongoing learning, and self organization

Task: Define Business Need

Task: Assess Capability Gaps

Task: Determine Solution Approach

Task: Define Solution Scope

Task: Define Business Case

Enterprise Analysis Agile Summary
All of the tasks in Enterprise Analysis are important on Agile projects. The primary distinctions in this Knowledge area are contributing to the continuous improvements of the organizations ability to deliver as well as the emergence of the business need through the project.

Today I am discussing the tasks from the BABOK associated with Eliciation. These are the tasks to Understand the underlying needs rather than stated or superficial desires. There are four tasks associated with Elicitation. Again, I show the BABOK® purpose. Then I discuss the impact on who and how in projects leveraging progressive elaboration, incremental delivery, close customer engagement, ongoing learning, and self organization.

Task: Prepare for Elicitation

Task: Conduct Elicitation Activity

Task: Document Elicitation Results

Task: Confirm Elicitation Results

Elicitation Agile Summary
All of the tasks in Elicitation are important on Agile projects. Again, the primary distinctions in this Knowledge area are planning for  continuous engagement with  the customer and the progressive elaboration of requirements. In the event that the enterprise has governance requirements or there are contractual obligation, the team will have to negotiate how they will deliver documentation that is sufficient to support the needs of the business.

Today I am discussing the tasks from the BABOK associated with Business Analysis and Planning. These are the tasks to determine which business analysis activities are necessary to support an effort. There are six tasks associated with Business Analysis and Planning. I show the BABOK® purpose. Then I discuss the impact on who and how in projects leveraging progressive elaboration, incremental delivery, close customer engagement, ongoing learning, and self organization.

Plan Business Analysis Approach

Conduct Stakeholder Analysis

Plan Business Analysis Activities

Plan Business Analysis Communication

Plan Requirements Management Process

Manage Business Analysis Performance

Business Analysis Planning and Monitoring Summary

All of the tasks in Business Analysis Planning and Monitoring are important on Agile projects. They won’t be dictated from existing policy, they will be decided on as a team. The activities will support the Agile approach. The primary distinctions in this Knowledge area are planning for  continuous engagement with  the customer and the progressive elaboration of requirements. In an enterprise that has governance requirements, the team will have to negotiate how they will deliver communications to the governance team that is sufficient to support the needs of the business.

Agile methods are helping teams deliver software faster and with much higher quality than ever before. In response, Agile has emerged as a driving force in software development. Businesses want to achieve the benefits of improved quality and rapid delivery.   But getting faster at development is just part of the equation.  As I discussed in my presentation “Feeding The Agile Beast”, you have to identify, prioritize, and scope what you are going to build so that risk is managed and ROI is maximized.

So, What’s New?

Agile methods have emerged from a few simple concepts. Change happens and people trump process (Thanks to Ken Schwaber via Steve Adolph). In response to this, substantially all of the Agile methods describe a way of delivering software that support the following attributes:

• Progressive elaboration a little ahead of demand (for planning,  requirements,  architecture, testing,  etc)
• Incremental and iterative delivery (though not time-boxed in Kanban/Lean continuous flow)
• Ongoing learning through close customer engagement regarding product fit/business value that informs requirements and prioritization
• Ongoing learning regarding development/delivery approach and interactions that informs ongoing improvement efforts
• Self organization around the work (where the people performing the work decides how best to do their jobs within organizationally appropriate constraints)

In traditional software development, this identification and prioritization of what you are going to build falls in the role of the Business Analyst. However, traditional business analysis methods do not operate in harmony with the iterative and incremental cadence of Agile methods. For the most part, this is different than traditional plan driven projects.  Over the next week or so, I am going through the capabilities associated with Business Analysis and discuss the how Agile impacts how business analysis is performed.

The Guide to the Business Analysis Body Of Knowledge

I am going to build this discussion on work from The Business Analysis Body of Knowledge® (BABOK®) from the International Institute of Business Analysis. The IIBA® is the independent non-profit professional association serving the growing field of Business Analysis.  The IIBA has collected the current generally accepted practices within Business Analysis in The Business Analysis Body of Knowledge® (BABOK®).The BABOK® Guide describes Business Analysis areas of knowledge, their associated activities and the tasks and skills necessary to be effective in their execution.

The BABOK has six Knowledge Areas.

• Business Analysis Planning and Monitoring: Determine which business analysis activities are necessary to support an effort.
• Elicitation: Understand the underlying needs rather than stated or superficial desires.
• Enterprise Analysis: Understand what the enterprise is capable of performing.
• Solution Verification and Validation: Determine which solution best fits the business need – includes assessing the performance and effectiveness of the solution.
• Requirements Analysis: Organize, prioritize, and model requirements and risks.
• Requirements Management and Communication: Ensure knowledge gained through business analysis activities throughout the effort is shared among the team.

Within each Knowledge Area are several tasks. I will focus on how progressive elaboration, incremental delivery, close customer engagement, ongoing learning, and self organization impact who performs business analysis tasks and how business analysis tasks are performed.

Agile Business Analysis

Just to set my context. All of the Capabilities that are reflected in the BABOK happen on all projects – traditional or Agile. They may or may not be done by a specific role. They may or may not happen in a specific phase. They may or may not be done with any ceremony or intention. They do occur. Since the identification, prioritization, and scoping of requirements is so important to success, they need to be done well. And on an Agile project they need to be done so they enable – not constrain or impede – the delivery of value in projects leveraging progressive elaboration, incremental delivery, close customer engagement, ongoing learning, and self organization.

This post is going to go back over the past month’s posts and bring them together into a model I can use to discuss Big Agile. I took me a while to noodle this out so I haven’t posted in over a week. This approach is based on over two decades of improving performance in development organizations. The last decade has been spent working to explicitly articulate what I’ve done to consistently deliver results. It is amazing how much you learn when you take concepts practiced unconsciously and write them in a (relatively) concise format.

Enterprise Agility

Many companies are dependent on technology to be competitive. Either because they deliver technology as their product or because technology enables the processes they depend on. For those companies it is important to be able to operate faster than their competition.  This requires not only the ability to rapidly deliver technology – but the ability to adapt their organization.  This is the concept of operating within your competitors decision cycle. The model I use to demonstrate this concept is John Boyd’s O-O-D-A model.

This is Enterprise Agility, when the enterprise has learned to leverage technology and change management to develop an energized workforce frequently delivering value that meets the emerging needs of the customer.

Three Concepts for Achieving the Agile Enterprise

I am going to build on three concepts in showing how to build Enterprise Agility.

1. Theory of Constraints. You can’t get there all at once. It is necessary to decide where to focus. You have to develop the underlying capabilities from the bottom up. If you can’t produce quality product, the best ideas in the world will run into problems. The way of thinking about problems covered in the management system the Theory of Constraints provides the method for deciding where to focus. Identify the current constraint, get the most out of the capability that is the constraint, subordinate the system to the constraint, elevate the constraint, and then continue to pursue this cycle. This “Process of On-Going Improvement” is applied to the flow of value to the customer.

2. Creative Solution Finding. Once we understand where to change we need to understand what to change to. We have to apply a method to lead the overall adoption effort that supports aligning the capabilities with the overall goals of the organization. The stages of Creative Solution Finding are useful for identifying the situation specific approach to developing this road-map. There is no magic sauce for developing this road-map. By applying the seven principles, along with a knowledge of applicable best practices, a useful road-map that fits both the process and social context of the Enterprise can be developed.

3. The Learning Organization. Technology development efforts are knowledge based efforts. As such, they are performed through a social construct. To improve the performance of software development we have to not only improve the methods of development, we have to improve the performance of the social construct. First the individuals on the team must be willing to change. While the People Design principle in Creative Solution Finding will help prepare individuals for change, their personal fears and concerns must also be addressed. Once the constraint is identified and a road-map for improvement is developed, the learning organization concepts in the Fifth Discipline are relevant. The right individual competencies must be developed on the team to do the job (Personal Mastery and Mental Models).  The team has to be functioning well together (Shared Vision and Team Learning).  And the Team has to be functioning in a way that is aligned with the goal of the overall system (Systems Thinking).  The primary tool used to influence a social construct to create change and to improve performance is the conversation.

Discussing Big Agile

So the approach is to take the Big Agile capabilities, (1) provide insight to identify when a specific set of capabilities at a level of scaling are the current constraint, (2) leverage the solution finding model and best practices to define/refine the roadmap to move toward,  (3) and then identify the competencies to develop and conversations that are necessary to execute the change.  Then continue the cycle continuously.

This is the Big Agile improvement approach. It takes a model that I have been applying for two decades implicitly and evolving explicitly for the last decade. Clear direction combined with appropriate best practices and an effective adoption approach. All that’s left now is to build out the details of the capabilities, when they are a constraint, how to select appropriate best practices, and the conversations and change approach at each level of scaling for the model.