Posts Tagged ‘Agile Quality Assurance’

My Job is hard, Yours is easy

Posted on August 30th, 2010 by Dennis Stevens  |  1 Comment »

I really enjoyed Agile 2010 this year. My workshop, “Feeding the Agile Beast”, and presentation, “Using Lean and Agile to Lead Business Transformation”, were both well received. We found good interest in the Agile Extension to the BABOK. ICAgile was a topic of conversation and for the most part we found an appetite for a solid certification model. I also got to spend time with people I consider to be in my tribes. Just to name a few – note: if I fail to mention someone it is not intentional, just tweet me and I will add you to the appropriate clan(s)

While some of these people, like myself, may have come from coding backgrounds, none of these people are primarily heads down technical resources. All of these people have spent years – some of us decades – improving our craft. All of these people bring deep experience, passion and knowledge to an area that is deeply necessary to deliver Agile projects to organizations.

It isn’t just about code

I was rankled by comments that came out at the end of the conference from people that I deeply respect that there wasn’t enough technical content at the conference. It wasn’t so much the comments as the context that they were delivered in. The troubling perspective was encapsulated in a post that said, “too much of the content was not technical. It was simple content that could be easily understood and digested.” I have tremendous respect for developers. I used be one – and I was pretty good. But about 15 years ago or so I realized that I could create more value in organization’s by removing the organizational constraints that create difficult environments for developers.

When you get beyond pretty small organizations, just writing code is not enough. In fact, without the capabilities practiced by my Agile Project Management, Kanban, Enterprise Agile, Agile Business Analysis, and Agile Testing clans, you can’t deliver value to customers. And the craft of being great at these in organizations is just as challenging, evolving, and important as any line of code anyone ever wrote.

Excuse me, but are you kidding me

Does anyone really think that the work of these clans is easily understood and digested? Getting organization’s to focus on strategy, then translate the strategy into work that can be developed, then coordinate that development across functions, then ensuring that what is needed is what is delivered, and then implementing the changes necessary to ensure the business receives value is not easily understood and digested. As a matter of fact, I think it is at least as interesting and difficult a problem as writing good code. And I think in the context of leveraging Agile software development in organization’s it is just as valuable and not as well understood.

My Job is Hard, Yours is Easy

A common bias in organization’s is the fundamental attribution error. This is also known as the self-serving bias. This manifests itself when people attribute different circumstances to their experience than they do to the experience of others. I see this all the time when coaching organization’s.  People devalue the contribution of others and overvalue their contribution. This attitude leads to bunkering down in silo’s, organizational conflict, and obstacles to improvement and flow of value.

I have written code in an organization. It is hard. But I have also been responsible for business analysis, project and program management, quality assurance, process improvements, training, organizational design, and change management aspects of projects. All of these are important, all of them are hard. Delivering value for the business requires cadence and flow among these disciplines. As you are making assessments of how simple and easily digestible non-technical content is remember this tendency. Just saying, “My job is hard, and yours is easy”, doesn’t make this assertion true. And it results in value destroying behavior.

We are Doing QA All Wrong

Posted on August 23rd, 2010 by Dennis Stevens  |  17 Comments »

Over the last month I have presented or participated in several user groups and conferences. There was Southern Fried Agile in Charlotte, Agile 2010 in Orlando, a PMI Atlanta professional growth event, meetings of Agile Atlanta and the Atlanta Scrum Meet-up, and Product Camp Atlanta (not surprisingly in Atlanta). These conferences give me an opportunity to meet with people doing, or attempting to do agile in the enterprise. There are a lot of common patterns that I am seeing. One of the most common patterns is that we are doing QA all wrong in most companies.

Imagine if someone approached you up front and said “I have an idea. Let’s design our product development system like this. First, development is going to take in less then precise requirements. Then development is going to build something from the requirements. Then we are going to deliver this to a separate group that will then figure out how to test it for accuracy. Then that team will test it. When the product doesn’t pass the test we just defined we will return it to development for them to fix it.”   Is this the way that your test organization operates with your development organization? Does the relationship between QA and development seem reasonable? Why are challenges between QA and development so prevalent in our organizations?

We Don’t Understand Quality Assurance

Quality Assurance and Quality Control are different things. We have Quality Assurance organizations – but most often they are involved in Quality Control control activities.

Quality Control: Find defects. These activities are focused on validating the deliverable. They are performed after the product (or some aspect of the product) has been developed. It also enforces fit to specification. This protects production and our customers from getting defective deliverables that do not meet the specification. But when done on a completed product it isn’t doesn’t enable flow and defects found downstream of the deliverable result in rework in development.

Quality Assurance: Prevent defects. These activities are focused on the process used to create the deliverable. They are performed as the product is being developed. So, Quality Assurance should be helping us improve our processes so we don’t create defects in the first place. A key point to this is that creating a common understanding of the specification and how we will validate it up front is really important to eliminating defects.

If your Quality Assurance activities are primarily focused on finding defects and ensuring fit to requirements after the fact then your QA group is primarily doing Quality Control. Getting QA involved in improving the specification communication process through is a good place to start. A specific technique might be exploring communicating acceptance criteria to development with the same examples QA will use to perform acceptance tests.

We Don’t Understand the Cost of Rework

Finding defects after the product has been delivered ensures that you will generate rework. Rework from failed QA tests are extremely expensive to flow. Sending work back into development interrupts work in process. This has a dramatic effect on cycle time across the system – which results in increased cost of every item in the system.

After the fact testing is also typically done in batches when a release or iteration is delivered to QA from development. Defects uncovered in these batches create bottlenecks at development.  Just like when traffic backs up on the highway, bottlenecks in product development result in a long term cost to the overall system.

We can dramatically reduce the cost and adaptability of the system by reducing this rework. One way to address this defect bottleneck is to perform quality control frequently – not in batches. Another way is to prevent defects through effective quality assurance in specification understanding and in development. This will reduce the number of defects produced that escape to Quality Control.

We Have Designed the Product Development System Wrong

We used to teach that there needed to be a separation of testing and development. Having them both report to the same person was like having the person that wrote the checks in accounting also reconcile the check book. It was too easy to cheat.  We created QA teams that were completely separate from development – and that reported up through different management structures. I was talking to the director of development at a company where the QA and development chains of command meet at the CEO. When you design the organization to QA and development in different silo’s you will make it very difficult to have effective Quality Assurance involvement. You will ensure Quality Control behavior.

This separation made sense at one point (I think). It doesn’t make as much sense today – especially as the resulting silos result in slower delivery times and increased costs. What has changed in the last 10 years is:

  • the cost of setting up testing environments is much lower
  • our ability to rapidly and frequently produce testable deliverables for these environments is much greater, and
  • our ability to perform acceptance, integration, regression, performance, load, and stress testing frequently is enhanced.

Quality Assurance needs to be part of the development team and process. We need to change our thinking around this.

So we need to integrate Quality Assurance very early in the process. We need to ensure early arrival of acceptance test examples to development. But, we still need independent test teams when we are doing large or complex projects that require integration of many team’s outputs. Independent test teams can be running parallel testing or test-immediately-after delivery. When they are getting high quality components, they focus on user experience, performance, load, and stress testing.  Move everything you can into the development teams (enhanced with quality assurance) so you can produce low defect escape rates. Then eliminate the defect bottleneck and increases in cycle time by treating most of the defects at the independent test team as new specifications to development.

We have the Wrong Expectations

We Expect Perfection – Not Improved Value

Our QA organizations tend to be focused on defect identification and perfection in delivery. The focus of QA, and everyone in the organization, needs to be on producing value-add and delighting the customer – not on defect-free code. This doesn’t mean that defects are okay – but every application has some defect or limitation. Unless you are building aircraft navigation systems or other critical systems you are going to deliver some defects. Make sure the defects don’t block business value – but there is a point where it is more important to the customers and business to deliver value.

We Expect Local Optimization – Not Flow

Rather than perfecting Quality Control, QA can add a lot of value helping the organization enhance the flow of value through the entire product development system. Delivering a unit of value requires analysis, development and testing. There is no value delivered internally. The local optimization problem of improving one area at the expense of another is not useful. Crossing the boundaries between analysis, development, and testing is needed to optimize flow of value. The various teams need to figure out how to do it together – not just improve their area.

Expect our Current Constraints to be Permanent

Organizations labor under the belief that their constraints are fixed. Find a way to break down the barriers that lead to your constraints. Don’t accept the current constraints as permanent. Increase thinking at the system level to find ways to improve the product development process.

We Don’t Use the Tools Available to Us

There is an impressive amount of automation and related techniques available at very low cost to help build the QA environment that allows for fast feedback, building on progressively tested levels, and improving opportunities for frequent testing. Developers should be using tools that support automated unit testing and only checking in code that passes all their unit tests. This provides a basis for validation and refactoring. Test driven development or test  just after development should be ubiquitous – but it is not. Continuous Integration environments that ensure that each check-in results in a valid and testable platform help teams perform integration and build validation. This helps everyone work on a stable build and ensures that no one gets way off track. There are automated tools for acceptance testing and some levels of UI testing. When the tools are combined you can build a very powerful regression testing environment that can also rapidly perform performance, load, and stress testing.

The lack of general adoption of these tools probably has multiple roots. Sometimes it can be difficult to leverage all of them with legacy systems. They require new skill sets and generate new artifacts that have to be maintained. On the surface, they automate tests that are currently performed by QA groups and so their is fear and organizational threat. But some subset of these tools should be broadly adopted by each team. QA teams can focus their efforts on valued added items like Exploratory Usability Testing and Release Validation.


Most organizations are doing QA all wrong. It is time to start doing it right, the cost of low quality in dollars, time, customer satisfaction, and the engagement of employees depends on it.

  • Switch their primary focus to Quality Assurance
  • Help the organization reduce re-work
  • Integrate QA and QC with analysis and the development organization and work to eliminate defects – not just identify themF
  • Enable value, flow, and ongoing improvement
  • Leverage proven tools and techniques across the Product Development Organization to increase the value delivered

The status quo is not sufficient to meet the needs of most organizations. While not all of these approaches will be right for every situation, the rate of adoption is too low to be justified. There is far too much latent value being tied up in organizations because we are doing QA all wrong.. What obstacles do you see in your organizations?