written by Rex Black

Back in the early days of computing, there was no such specific, separately identifiable activity known as "testing." Developers debugged their code, and that was usually intertwined with some unit testing task. That didn't work. My first job was as a programmer, and where I worked, this was exactly how we approached quality assurance. It was a quality disaster.

This approach by itself still doesn't work. It does not work for those throwback, Neanderthal organizations that rely on this approach entirely. It does not work for those cutting-edge companies that think some fancy language or process or tool has solved the software quality problem at last. It does not work for anyone, unless we define "work" as "shift most costs of poor quality onto end users who are too stupid to know better or trapped in a monopolistic market without any real choice." And those organizations that rely on stupid users or a monopoly position had better hope they are right.

With the widespread advent of independent test teams in the late 1980s and early 1990s, we saw improvements. I was working in an independent test lab in the late 1980s and as a test manager in the early 1990s, and we made great strides. However, we also saw the emergence of a new dysfunction, the "hurl it over the wall to the test guys and hold them responsible for delivered quality" mistake. Every now and then, we work with organizations that still suffer from this problem.

Let's be clear. When quality matters-and shouldn't it always-everyone must play a role. Good testing involves a series of quality assurance filters. Each team in the organization typically participates in and owns one or more of these filters.

Some of these filters, especially high-level testing like system test and acceptance test, work best with a high level of independence. Some of these filters, especially low-level testing like unit testing, work best with less independence. Let's survey the degrees of independence so you understand your options.

Self-testing occurs when the developers test their own code. There is no independence here, of course. The author bias problem is significant, and the developers-even if given enough time to do unit testing-often miss the important bugs because they determine that the code works as they intended. Of course, that might not match the actual requirements. The advantages are that the developers can fix any defects they find quite quickly and, being quite technical, understand the software being tested.

Buddy testing occurs when developers test each other's code but not their own. Pair programming, which is a practice in some agile techniques, is a special form of this, where development of code, continuous code review, and development and execution of unit tests by a team of two programmers evolves the code. While the author bias problem is not so acute here, when two people work closely together, it's hard to say there is much independence between the developer and the tester. In addition, there tend to be few if any usable defect metrics captured in this situation without careful cultivation of the proper mindset because when peers test each other's code, they might not want to report defects. Finally, since the average programmer has little training or formal experience with testing, the mindset is usually focused on positive tests. Once again, the advantages include a quick repair of defects and good tester understanding of the software being tested. 

Having the tester or testers inside of development occurs when a development team includes one or more testers and these testers are not part of an independent team. This is rather popular these days because many proponents of agile methodologies advocate this approach. There is nothing wrong with it as part of a larger quality assurance process, but by itself it can be dangerous. The main problem is editing and self-editing.

Self-editing means that the tester does not report-or reports only informally to the developer-problems they find, leaving no official trail in a bug tracking system. This is the equivalent of an organization tearing out its eyes and flying blind with respect to quality. Defect metrics are necessary to any balanced, meaningful picture of quality. In addition, an organization that doesn't study its mistakes is unable to learn from them.

Even if the tester does report bugs, editing can happen. The development manager or project manager does not allow the tester to release a clear, balanced, complete set of test results to the broader set of stakeholders. Furthermore, since the development manager or project manager is often focused on short-term goals like getting the product released on time and on budget, the entire mission for the testers is likely to be verifying adherence to requirements. Finally, it is often the case in these arrangements that the tester tasks are assigned to junior developers or factotums of some sort or another, along with a number of other responsibilities. Therefore, the testing is often done hurriedly and without any particular professionalism.

All those disadvantages enumerated, I do see value in having one or more testers-whose permanent positions are inside an independent test team and who are true professional testers-assigned to act as testing resources within development teams. In this role they can create good test cases, build automated test harnesses, create continuous integration build-and-smoke-test facilities, and the like. We have played this role for clients in the past, with great success. However, this too is often not sufficient by itself.

Visit our articles page to read this article in its entirety.

Mark Your Calendar.....

Webinar attendees are automatically entered into a drawing to win their choice of one of our green e-learning courses.  Visit our training page to see the complete webinar schedule, or just look on this email, sign up for a webinar, show up at whichever webinar session is most convenient, and--who knows--you might be the lucky winner of some valuable free training.  Either way, you're sure to learn something. 

Congratulations Ronald Reynolds, an attendee of the July webinar, for being selected as the winner of an e-learning course.

Register now for our next complimentary webinar, "Strategies of Testing, Not Schools", September 4 or 5, 2014.