Imagine if someone approached you up front and said “I have an idea. Let’s design our product development system like this. First, development is going to take in less then precise requirements. Then development is going to build something from the requirements. Then we are going to deliver this to a separate group that will then figure out how to test it for accuracy. Then that team will test it. When the product doesn’t pass the test we just defined we will return it to development for them to fix it.”   Is this the way that your test organization operates with your development organization? Does the relationship between QA and development seem reasonable? Why are challenges between QA and development so prevalent in our organizations?

We Don’t Understand Quality Assurance

Quality Assurance and Quality Control are different things. We have Quality Assurance organizations – but most often they are involved in Quality Control control activities.

Quality Control: Find defects. These activities are focused on validating the deliverable. They are performed after the product (or some aspect of the product) has been developed. It also enforces fit to specification. This protects production and our customers from getting defective deliverables that do not meet the specification. But when done on a completed product it isn’t doesn’t enable flow and defects found downstream of the deliverable result in rework in development.

Quality Assurance: Prevent defects. These activities are focused on the process used to create the deliverable. They are performed as the product is being developed. So, Quality Assurance should be helping us improve our processes so we don’t create defects in the first place. A key point to this is that creating a common understanding of the specification and how we will validate it up front is really important to eliminating defects.

If your Quality Assurance activities are primarily focused on finding defects and ensuring fit to requirements after the fact then your QA group is primarily doing Quality Control. Getting QA involved in improving the specification communication process through is a good place to start. A specific technique might be exploring communicating acceptance criteria to development with the same examples QA will use to perform acceptance tests.

We Don’t Understand the Cost of Rework

Finding defects after the product has been delivered ensures that you will generate rework. Rework from failed QA tests are extremely expensive to flow. Sending work back into development interrupts work in process. This has a dramatic effect on cycle time across the system – which results in increased cost of every item in the system.

After the fact testing is also typically done in batches when a release or iteration is delivered to QA from development. Defects uncovered in these batches create bottlenecks at development.  Just like when traffic backs up on the highway, bottlenecks in product development result in a long term cost to the overall system.

We can dramatically reduce the cost and adaptability of the system by reducing this rework. One way to address this defect bottleneck is to perform quality control frequently – not in batches. Another way is to prevent defects through effective quality assurance in specification understanding and in development. This will reduce the number of defects produced that escape to Quality Control.

We Have Designed the Product Development System Wrong

We used to teach that there needed to be a separation of testing and development. Having them both report to the same person was like having the person that wrote the checks in accounting also reconcile the check book. It was too easy to cheat.  We created QA teams that were completely separate from development – and that reported up through different management structures. I was talking to the director of development at a company where the QA and development chains of command meet at the CEO. When you design the organization to QA and development in different silo’s you will make it very difficult to have effective Quality Assurance involvement. You will ensure Quality Control behavior.

This separation made sense at one point (I think). It doesn’t make as much sense today – especially as the resulting silos result in slower delivery times and increased costs. What has changed in the last 10 years is:

• the cost of setting up testing environments is much lower
• our ability to rapidly and frequently produce testable deliverables for these environments is much greater, and
• our ability to perform acceptance, integration, regression, performance, load, and stress testing frequently is enhanced.

Quality Assurance needs to be part of the development team and process. We need to change our thinking around this.

So we need to integrate Quality Assurance very early in the process. We need to ensure early arrival of acceptance test examples to development. But, we still need independent test teams when we are doing large or complex projects that require integration of many team’s outputs. Independent test teams can be running parallel testing or test-immediately-after delivery. When they are getting high quality components, they focus on user experience, performance, load, and stress testing.  Move everything you can into the development teams (enhanced with quality assurance) so you can produce low defect escape rates. Then eliminate the defect bottleneck and increases in cycle time by treating most of the defects at the independent test team as new specifications to development.

We have the Wrong Expectations

We Expect Perfection – Not Improved Value

Our QA organizations tend to be focused on defect identification and perfection in delivery. The focus of QA, and everyone in the organization, needs to be on producing value-add and delighting the customer – not on defect-free code. This doesn’t mean that defects are okay – but every application has some defect or limitation. Unless you are building aircraft navigation systems or other critical systems you are going to deliver some defects. Make sure the defects don’t block business value – but there is a point where it is more important to the customers and business to deliver value.

We Expect Local Optimization – Not Flow

Rather than perfecting Quality Control, QA can add a lot of value helping the organization enhance the flow of value through the entire product development system. Delivering a unit of value requires analysis, development and testing. There is no value delivered internally. The local optimization problem of improving one area at the expense of another is not useful. Crossing the boundaries between analysis, development, and testing is needed to optimize flow of value. The various teams need to figure out how to do it together – not just improve their area.

Expect our Current Constraints to be Permanent

Organizations labor under the belief that their constraints are fixed. Find a way to break down the barriers that lead to your constraints. Don’t accept the current constraints as permanent. Increase thinking at the system level to find ways to improve the product development process.

We Don’t Use the Tools Available to Us

There is an impressive amount of automation and related techniques available at very low cost to help build the QA environment that allows for fast feedback, building on progressively tested levels, and improving opportunities for frequent testing. Developers should be using tools that support automated unit testing and only checking in code that passes all their unit tests. This provides a basis for validation and refactoring. Test driven development or test  just after development should be ubiquitous – but it is not. Continuous Integration environments that ensure that each check-in results in a valid and testable platform help teams perform integration and build validation. This helps everyone work on a stable build and ensures that no one gets way off track. There are automated tools for acceptance testing and some levels of UI testing. When the tools are combined you can build a very powerful regression testing environment that can also rapidly perform performance, load, and stress testing.

The lack of general adoption of these tools probably has multiple roots. Sometimes it can be difficult to leverage all of them with legacy systems. They require new skill sets and generate new artifacts that have to be maintained. On the surface, they automate tests that are currently performed by QA groups and so their is fear and organizational threat. But some subset of these tools should be broadly adopted by each team. QA teams can focus their efforts on valued added items like Exploratory Usability Testing and Release Validation.

Summary

Most organizations are doing QA all wrong. It is time to start doing it right, the cost of low quality in dollars, time, customer satisfaction, and the engagement of employees depends on it.

• Switch their primary focus to Quality Assurance
• Help the organization reduce re-work
• Integrate QA and QC with analysis and the development organization and work to eliminate defects – not just identify themF
• Enable value, flow, and ongoing improvement
• Leverage proven tools and techniques across the Product Development Organization to increase the value delivered

The status quo is not sufficient to meet the needs of most organizations. While not all of these approaches will be right for every situation, the rate of adoption is too low to be justified. There is far too much latent value being tied up in organizations because we are doing QA all wrong.. What obstacles do you see in your organizations?

Don’t Support Progressive Elaboration

One of the faulty assumptions made in Requirements Elicitation is that we believe we can know up front exactly what we want. It is also faulty to believe that we should document requirements in too granular detail. In fact, neither of these is accurate. Requirements should support the concept of progressive elaboration. That is, they should be documented so that more specific detail unfolds during the course of the project. This addresses the flaw in both of assumptions. First, documenting what we know at a high level of detail allows us to establish a consistent context that we can communicate in more detail the more we learn. Getting to too much detail early can result in rework and inconsistency in communication.

Agile teams may break requirements into Themes, Epics, Stories, and then Tasks. This may be hierarchical, follow a story-map, or be connected in a mind-map.  Themes are be defined early on and directly connected to business value propositions. These are then progressively elaborated into Epics and Stories as needed to feed the team. As Stories move into a ready state for development they can be further broken down into tasks by the development team. This progressive elaboration allows the team to apply what it learns about the best way to deliver the project as the project unfolds. Critically, it also supports the way that people learn about the problem. Too much detail early on can be overwhelming and result in a disconnected or inconsistent understanding of the effort. Not enough detail later on results in a lack of shared understanding of the problem. Progressive elaboration supports effective communication, collaboration in solution finding, and the application of what the team learns as the project unfolds.

“It is not about working software. It is not about your preferred working process. It is not about user stories. It is not only about your customer or user. It is all about delivering, to your set of Stakeholders, value improvements to them, to what they care about, what makes their world better, within agreeable, minimum or pre-determined costs. If that is not the main focus, if that is not clear to everyone on the team, you will eventually loose! If your methods are not focusing on delivering that value, as todays most popular Agile methods are not, they will fail to deliver, they will become last years fad.”

They claim Agile software development is flawed because it focuses on improving the process and environment of the team developing software. These improvements result in improved customer collaboration that ensures the team better understands what the customer wants. From my experience, the combination of improved environment, appropriately sized process, and customer collaboration results in mature agile teams doing a great job of delivering the projects and products. But I understand their point. There is a difference between understanding the customer and focusing on delivering customer value.

Wrong Focus

Whether the software development organization is building software for external clients or to enable the organization’s business processes, there are always more things that can validly be improved than there are resources to do the work. Not all of the improvements will result in the same return of business value to the overall organization. Some benefits are more valuable to the organization than others. If you don’t pick the one that is most valuable to the business it is not the right investment for the business. In any of these improvements, there is more improving that is valid than is necessary to achieve the benefit. If the team works on improvements that aren’t the focus of the effort, “while they are there”,  they are not focusing on business value. This delays delivery of the effort and is not the right investment for the business.

Nothing in the Agile Manifesto, nor most of the Agile methods, give the tools to pick the right improvements to focus on. They also create an environment where it is extremely likely to work on more then is the minimum necessary to achieve the targeted business benefit. In many organizations, close customer collaboration can lead to “gold-plating” of features. If you offer the customer whatever they want – they will ask for it.

Wrong Focus in Practice

I have a customer right now that is going to build some software to support their field employees. The business goal is to achieve a significant cost reduction in the field through improving quality and efficiency in the field. The developers are extremely interested in developing mobile applications, distributed collaboration, GPS interfaces, image capture and markup, mapping, barcode, and training training. They have started a new business to build this product with the hope of selling it outside the business in the future. The challenge here is how do they clearly identify those improvements in the field and in dispatching that will deliver the significant reduction in field expense. If they go and and talk to the dispatchers they might find 30 things that could be done better. All of these improvements would result in an improvement in work conditions or performance. But they may not be aligned with the current business focus. The same will happen in the field.

Just because it will improve satisfaction or make someone’s job easier doesn’t mean it will help with the current business focus. How do they align absolutely the minimum amount of technology to deliver on these improvements? Especially when their interests are in this bigger, more robust solution. Especially when this limiting of features is not in line with their future business objectives. Nothing in Agile addresses how to deal with this conflict.

Where’s the Answer

I address this in Feeding the Agile Beast. This is the domain of Business Analysis. As the Gilb’s point out, most Agile methods abstract this from the team. They focus on improving their processes and collaboration. In Scrum, Business Analysis is abstracted behind the Product Owner. In XP, it is abstracted behind the Onsite Customer. I don’t agree with the implication from the Gilb’s that this abstraction is a deficiency in Agile. It is just outside the scope. I also don’t agree with Agile teams that close collaboration with the customer is sufficient to ensure business value is achieved. This is not an OR conversation. This needs to be handled as an AND conversation. This is the next thing we need to learn – how to keep an Agile team laser focused on business value AND maintain the benefits of understanding the customer that comes with Agile software development.

BABOK® is consistent with this. It defines a requirement as:

1. A condition or capability needed by a stakeholder to solve a problem or achieve an objective
2. A condition or capability that must be met or possessed by a solution or solution component to satisfy a contract, standard, specification, or other formally imposed documents.
3. A documented representation of a condition or capability as in (1) or (2).

The purpose of documenting requirements is not to document detailed steps that someone performs – it is to communicate the outcome needed to achieve an objective. One problem that arises is when requirements are expressed as a document of the existing process – typically through a review of existing process documentation or through observation of people doing the job. As I have discussed in previous posts, just documenting process can lead you into the How Trap and it can fail to generate a useful shared understanding. These problems arise when the focus is on how stakeholders are currently doing the work and not on understanding the challenges, problem or objective.  But, focusing on process is a problem in another way. It is an unstable lens.

Unstable

Modeling processes early in the elicitation process is not helpful to creating an appropriate understanding of the effort. Process reflects how people say they ought to do a task in perfect circumstances. The work is almost never consistently performed based on process documentation – or even how you observe the process being performed. Exceptions, reorganizations, changes in personnel within or adjacent to process, new tools and technologies, and innovation will change the process view. Even just asking five different people what the process is will likely produce five different results.

The process view can also propagate flawed understanding on the part of the people doing the work. You have likely heard the story about the recently married young women who was cooking her first Thanksgiving dinner in her own home. She had invited her family over the beautiful house she shared with her husband to show her parents how successful she had become and to demonstrate her gratitude to her parents. As she brought out the turkey for dinner her mother noticed the young women had cut the tail off of the turkey. The mother asked her daughter why she had cut the tail off the turkey. “Because that’s they way you always did it, Mom.” The mother smiled and replied, “That’s because the only pan we owned was too small for the turkey.”

A process based view hides assumption, complexity and is an unstable representation. The real requirement doesn’t change based on who you ask, who is doing the job, or the most recent reorganization. Basing requirements on unstable views doesn’t reflect the intent of the solution. This may result in an expression of the solution that doesn’t meet the needs of the business. As you observe a process or review process documentation ask why each step is important and what the intent of each step is. Consider what would happen if the steps were done in a different order or something was left out. Creating a stable, purpose driven view of the objective will provide valuable input to the development organization that will result in an improved solution.

So, What’s New?

First off, the business analysis tasks are not just aimed at documenting requirements. There are three primary targets of analysis activities. Many tasks address the features of the product. These are aimed ensuring requirements are elicited, communicated, and the solution is verified. There are business analysis tasks aimed at understanding organizational readiness and transition requirements. Finally, there are business analysis tasks aimed at understanding and improving the ability of the organization to develop and deliver the solution.

Secondly, all the tasks are still necessary. However, unlike a traditional project, where many of the analysis tasks are performed up front, they are performed continuously throughout the project. The requirements for the solution, the ability to deliver, and the readiness of the organization are evaluated incrementally throughout the project and the approach is adapted to improve fit and performance.

Finally, the understanding of requirements, organizational readiness, and development capabilities is progressively elaborated throughout the project.  In Agile, the development team itself performs detailed business analysis activities within the development iterations. The business analyst continues to operate in front of and after the development efforts. Lightweight requirements documentation combined with an emerging understanding means the business analyst become an advocate for the product. They collaborate with the team to ensure a shared understanding unfolds. The business analyst also facilitates the understanding and implementation of improvements in the business analysis and organizational readiness processes.

Summary of Business Analysis Knowledge Areas

Business Analysis Planning and Monitoring.  The tasks in this knowledge area are still primarily performed on the front end. The team will agree on the approach it will take in business analysis tasks. This means agreeing on who and how. All the task purposes in the BABOK still need to be planned – but planned to support the cadence of the team and the emergence of understanding through the project. The way the business analysis tasks are performed are not cast in stone, they will be updated through inspection and adaptation throughout the life of the project.

Elicitation. This knowledge area occurs continuously in a progressive fashion and responsibility for the product requirements changes hands during each iteration.  These hand-offs associated with the various levels of elicitation can contribute to lost knowledge and increased transaction costs.  An approach that ensures continuity of understanding is critical.

Requirements Management and Communication. The existence of Requirements Management and Communication in a single Knowledge Area demonstrates the closely held traditional belief that documenting and tracking something is the best way to communicate it. It is important that the method of documenting requirements at the higher level is lightweight while still facilitating understanding, focus and priority. In Agile projects, communication should be viewed as completely distinct tasks. At the detailed levels, there is a transfer of context, purposes, and outcomes requirements that is best performed through conversation between the customer and the development team.

Enterprise Analysis. The business analyst will still carry primary responsibility for understanding organizational readiness, transition requirements, and solution verification.  They will need to communicate findings from releases back into transition requirements to improve release performance. Gaps in the fit of the solution will feed back into the requirements backlog. Enterprise Analysis is very strategic, outside the scope of the typical Agile development team, and critical to the successful realization of business value.

Solution Assessment and Validation. These tasks happen continuously in a progressive fashion. The Big Solution Architecture will still be developed with input from the Business Analyst as well as Architects. The Business Analyst will need to understand and address the impact that progressive elaboration has on the transition requirements.

Requirements Analysis. Prioritization and organization of requirements happens as part of backlog prioritization. The business analyst should make sure that business value concerns and risks associated with assumptions and constraints are clearly understood here. Like Enterprise Analysis these tasks are very strategic, often outside the scope of the typical Agile development team (presumed to be understood by the Product Owner), and are critical to the team’s ability to burn down risk early and get to business value fast.

Missing Tasks

I believe there are at least three areas regarding business analysis on Agile projects that are not addressed by the BABOK.

Enrolling Stakeholders involves making sure everyone understands how they will participate. Just documenting a process or developing some templates don’t ensure appropriate engagement.  Stakeholders may play a role making sure shared understanding of the product requirements exists, they may be responsible for development process feedback, or they may need to participate in the transition of the solution.

Understanding Requirements is the new task or set of tasks that deal with generating the shared understanding of context, purpose, and outcome that is necessary for successful projects. Since the understanding of what is being developed and how it is being developed will emerge over the course of the project specific tasks must be in place to establish this shared understanding.  This may sound like a subtle extension of communicate requirements but this is social in nature and is fundamentally different than the push of communicating requirements.

Organizational Learning extends Enterprise Analysis. Not only does the readiness of the organization to use the product need to be reviewed, the ability to develop, deliver, and support the product needs to be evaluated. This task includes the introspection, retrospectives, operations reviews, and outside review of best practices that the organization performs to improve its ability to use and deliver solutions.

Agile Business Analysis Summary

The common concept that Business Analyst’s add no value to Agile projects is flawed. It is true that some of the detailed business analysis activities will be best served through direct interaction between developer’s and the customer. The high level planning and coordination around these direct interactions should not be abandoned but should be managed to facilitate focus on business value and shared understanding while minimizing transaction costs.

Additionally, there are many critical business analysis tasks outside the scope of the development team. Particularly, the tasks involving Solution Validation, Enterprise Analysis, and the inputs to Requirements Analysis are very strategic, outside the scope of the typical Agile development team, and critical to the successful realization of business value. A clear understanding of the tasks and purposes in the BABOK® demonstrates the strategic nature of Business Analysis tasks, highlights where the tasks should be performed within the Agile development team, and will increase the ability of the development team to achieve its objective of rapid delivery customer value.

Requirements Management and Communication: BABOK® Tasks

Prioritize Requirements

Organize Requirements

Specify and Model Requirements

Define Assumptions and Constraints

Verify Requirements

Validate Requirements

Agile Requirements Management and Communication

All of the tasks in Solution Verification and Validation are important on Agile projects. The primary distinctions in this Knowledge area are the support for the emergence of the solution over time. Particularly interesting are Assess Organizational Readiness and Transition Requirements as these are often overlooked in Agile projects (many software development projects). Remember, the product isn’t done-done until the customer is using it and achieving the benefits expected by the product owner.

Assess Proposed Solution

Allocate Requirements

Assess Organizational Readiness

Define Transition Requirements

Validate Solution

Evaluate Solution Performance

Solution Verification and Validation

All of the tasks in Solution Verification and Validation are important on Agile projects. The primary distinctions in this Knowledge area are the support for the emergence of the solution over time. Particularly interesting are Assess Organizational Readiness and Transition Requirements as these are often overlooked in Agile projects (many software development projects). Remember, the product isn’t done-done until the customer is using it and achieving the benefits expected by the product owner.

Task: Define Business Need

Task: Assess Capability Gaps

Task: Determine Solution Approach

Task: Define Solution Scope

Task: Define Business Case

Enterprise Analysis Agile Summary
All of the tasks in Enterprise Analysis are important on Agile projects. The primary distinctions in this Knowledge area are contributing to the continuous improvements of the organizations ability to deliver as well as the emergence of the business need through the project.

Task: Prepare for Elicitation

Task: Conduct Elicitation Activity

Task: Document Elicitation Results

Task: Confirm Elicitation Results

Elicitation Agile Summary
All of the tasks in Elicitation are important on Agile projects. Again, the primary distinctions in this Knowledge area are planning for  continuous engagement with  the customer and the progressive elaboration of requirements. In the event that the enterprise has governance requirements or there are contractual obligation, the team will have to negotiate how they will deliver documentation that is sufficient to support the needs of the business.

Plan Business Analysis Approach

Conduct Stakeholder Analysis

Plan Business Analysis Activities

Plan Business Analysis Communication

Plan Requirements Management Process

Manage Business Analysis Performance

Business Analysis Planning and Monitoring Summary

All of the tasks in Business Analysis Planning and Monitoring are important on Agile projects. They won’t be dictated from existing policy, they will be decided on as a team. The activities will support the Agile approach. The primary distinctions in this Knowledge area are planning for  continuous engagement with  the customer and the progressive elaboration of requirements. In an enterprise that has governance requirements, the team will have to negotiate how they will deliver communications to the governance team that is sufficient to support the needs of the business.