Hi Cathy,
Great question. I'm a little leary of the
word "best," as every team is different and
you should achieve better results by trying things and adapting to your actual project than I will sitting here in my chair.
That said, I have been experimenting for several years with a variety of approaches to integrating QA and testers into agile teams, and I've found something that seems to work pretty well. This is the approach we put into the book. Warning: this post is a bit long and I'm afraid it just scratches the surface.
First, the goal of the team should be to produce software with no bugs. This is an ideal, but successful XP teams
have come close to this ideal. It's entirely feasible to get down to a few new bugs
per month.
To achieve this goal, the whole team--testers, customers, and programmers alike--need to take responsibility for quality. The goal is to reduce the number of bugs that escape the team rather than the number of bugs
found by testers. There are a number of practices to help you get there, covered in the "No Bugs" section of our book. Briefly, they are:
Practices to reduce programmer error:
Test-Driven Development - note that this is done by programmers and, when done well, results in a comprehensive regression test suite. Pair Programming Energized Work Coding Standards 80% of the defects are found in 20% of modules, according to Boehm. These practices help eliminate technical debt and eradicate bug breeding grounds:
Simple Design Incremental Design & Architecture Continuous Design Reflective Design & Refactoring Slack Test-Driven Development Pair Programming Collective Code Ownership Even when programmers work perfectly, bugs can result from programmer misunderstandings. These practices help prevent requirements defects:
On-Site Customers Active Customer Review Customer Tests Iteration Demos "Done Done" checklist So these are all things that the whole team is doing to
prevent bugs rather than find them. Testers bring a unique combination of customer-centricity and detail-oriented technical knowledge to the table. They provide both requirements-oriented and technically-oriented assistance.
On the requirements side, they help customers articulate their requirements and they identify gaps in the customers' statements. They help create customer tests, which are automated examples of how the business domain works.
On the technical side, they act as technical investigators for the team. They help provide answers to questions like, "How scalable is our software?" , "Is the software fast enough?", and "Is our software stable under load?" These questions are answered by creating long-running automated tests.
All of these practices together should lead to a team that produces nearly bug-free code. To check if this is true, testers also engage in
exploratory testing, a technique for rapidly creating and executing test plans. Each test plan builds on the knowledge gained from the previous. The purpose of exploratory testing isn't to create a regression suite (TDD provides that) but to rapidly explore and test a wide variety of scenarios.
Exploratory testing is very good at identifying unexpected problems. The purpose of exploratory testing in this process is not to find bugs, though, but to determine whether the team's process is working. When a bug is found, it's fixed, of course, but the team also conducts
root-cause analysis to determine how the bug slipped through the safety net provided by the practices. Then they adjust their practice using tools like
retrospectives to remove the hole.
When this is working well, the team no longer requires manual testing before release. It has practices that lead to nearly bug-free software and it has confidence (thanks to exploratory testing) that those practices are working. As a result, any build can be pushed to production without further testing.
For people who haven't experienced this level of confidence, the idea of pushing software to production without further testing seems very scary. Remember, though, the team
is testing, and testing thoroughly. Mature teams will have thousands of unit and integration tests and hundreds of customer tests. When done well, these tests run very quickly, so that the software can be built, tested, and deployed in less than ten minutes. (!)
To sum up, testers have an integral role on the team, but it's a very different role than seen in phase-based processes. Testing is done concurrently with development, not just before release or the end of the iteration, and it's quite thorough and rigorous.
