Resources
Our notable partnerships give us access to the best resources and tools for the job.
Get software testing training resources:
A metric is a measurement scale (e.g., Fahrenheit, defect severity) and a method used for measurement (e.g., a thermometer, a bug tracking tool). Defect metrics are metrics specifically focused on defects.
Resources for “Defect Metrics”
Articles
Mistakes lead to the introduction of defects (also called bugs). As I personally am aware, like all human beings I can make mistakes at any point in time, no matter what I might be working on. So it is on projects, where the business analyst can put a defect into a requirements specification, a tester can put a defect into a test case, a programmer can put a defect into the code, a technical writer can put a defect into the user guide, and so forth. Any work product can and often will have defects because any worker can and will make mistakes!
Now, I like to drink good wine, and I’ve collected bottles from around the world on my travels. Some I’ve stashed away for years, waiting for the right time to drink them, which always comes sooner or later. Some of these bottles have gotten a lot more valuable over the years. Odd as this will sound, in just a few ways, bugs are like wine. They definitely get more expensive with age, taking more effort and thus incurring more cost the longer they are in the system. Also, sooner or later most bugs need to be fixed. However, it’s definitely not a good idea to leave bugs lying around—or perhaps crawling around—in a cellar, and they’re certainly not appetizing!
Last time, I talked about an internal test process, managing test execution. This is a process that is only indirectly visible to the rest of the development team, in that, as long as you get through all your planned tests, effectively respond to change, and report your findings intelligibly, then for all most people care your testing could occur through voodoo, augury, and a Ouija board.
There are two measures that have a strong influence on the outcomes of software projects: 1) Defect potentials; 2) Defect removal efficiency.
The term “defect potentials” refers to the total quantity of bugs or defects that will be found in five software artifacts: requirements, design, code, documents, and “bad fixes” or secondary defects.
Aggregate defect data (bug reports), presented graphically, allow testers to give development, project, and executive management “dashboard” information they need to drive development projects to successful conclusions. I discuss four charts that distill meaning from test findings in terms of product stability, quality, bug repair, root cause, and affected subsystems.
Most software test teams exist to assess the software’s readiness prior to release. To achieve this goal, two primary tactics are used:
- Execute test cases or scenarios that are likely to find errors, resemble actual usage, or both.
- Report the test results, the defects found, and defects fixed, which, collectively, make up the test status and reflect the quality of the software.
For the test manager, the first task category primarily involves managing inward: assembling the test team and test resources, designing a solid test system, implementing the test system, and running the tests in an intelligent, logical fashion. The second area of responsibility involves upward and outward management. Upward management includes how you communicate with your managers, other senior managers, and executive staff. Outward management includes communication with management peers in development, configuration/release management, sales, marketing, finance, operations, technical support, and so on. As a test manager, your effectiveness in reporting test status has a lot to do with both your real and perceived effectiveness in your position. To put it another way, your management superiors and peers measure your managerial competence as much by how well you report your results as by how you obtain your results.
In a speech at Quality Week ’99, Roger Sherman, a Microsoft test manager, identified the leading cause of bug report closure as “unreproducible.” This is a regrettable circumstance, since such bug reports waste precious time during tight development schedules, add absolutely nothing to product quality, and lead to frustration and bad feelings between development engineers and test engineers. Sometimes, these bug reports arise from transient or random events, inconsistency of tools and configurations between test and development, or a vague definition of “correct” behavior under the tested conditions, but many bug reports closed as unreproducible are unclear, misleading, or just plain wrong.
Webinars
Webinar: “Teacher, Teacher, Bugs Ate My Deadline,” and Other Defect Management Disasters
Recorded March 19, 2019
Podcast Episodes
Why Does Software Quality (Still) Suck
Length: 1h 49m 0s
Software quality, for the most part, sucks. It still sucks, seventy-five years since the advent of the programmable computer. Software bugs are a constant fact of life, thanks to the ubiquity of software and the ubiquity of software bugs. Sometimes the bugs costs millions of dollars or kill people. Why is the reaction so muted? Rather than just accept software bugs as unavoidable, let’s ask the obvious question: Given that manufacturing is able to achieve six sigma levels of quality—i.e., only three defective items per million manufactured—why does software quality still suck? In this webinar, Rex will address some of the real barriers to achieving six sigma quality in software, while at the same time holding software engineering as a profession accountable for not doing nearly as much as we can.
One Key Idea: Two Simple, Useful Defect Metrics
Length: 0h 30m 44s
Technical debt is bad. It’s smart to carefully manage technical debt. Defects are a form of technical debt. Do you know how to measure how well you are managing defect-related technical debt? In this One Key Idea session, Rex will demonstrate two simple defect metrics, easily extractable from any defect management tool, which can give you useful insights into what’s happening with defect-related technical debt. In twenty minutes or less, you’ll learn what these metrics can tell you and how you can use them to manage your technical debt better.
Two Points of View at Two: Short High-impact Talks about Testing, with Michael Mah
In this session, Rex is happy to welcome Michael Mah. As managing partner of QSM Associates for over 20 years, Michael and his colleagues have assembled one of the largest worldwide databases containing software quality and productivity trends from over 10,000 completed projects. Michael and Rex will discuss defects, or bugs - and how in today’s competitive market, they can either make or break your company. We’ll discuss real world case studies and key findings from real world benchmarking and diagnostic engagements on what separates the leaders from has-beens. Michael and Rex have over 60 years of collective experience dealing with defects and their consequences, so they will discuss these important topics, and take your questions.
One Key Idea: Measuring Defect Detection Effectiveness
Length: 0h 10m 51s
A typical objective of testing is finding defects, but how effectively do you achieve that objective? Fortunately, there’s a way to measure that, using a metric called defect detection effectiveness. In this One Key Idea webinar, Rex will explain how to use two simple, easy-to-gather metrics to calculate defect detection effectiveness, and also share some insights into how to use that metric to set a baseline for future improvements and to benchmark yourself against other companies.
The Down Part of “Shift Left and Down”
Length: 1h 3m 36s
Shifting defect discovery to earlier activities (Shift Left) reduces rework and improves quality deliveries to customers. A lot of attention is paid to this part of “Shift Left and Down”, but how do we do the Shift Down? Root Cause Analysis (RCA) is the answer. A variety of techniques can be used with varying formalities, cost, and results; ranging from informal, 5-why, Ishikawa/fishbone to Cause-Effect. Which technique is the right one to use, and when should it be applied? Is your organization ready for RCA? What do you do with the results? These questions and others will be discussed by Ed Weller, who will draw on 20 years of experience using RCA and onsite training across a wide range of businesses.
Training
There are no training products in this category currently.
All Rights Reserved.
All Rights Reserved.
PMI is a registered mark of the Project Management Institute, Inc.