Resources

Our notable partnerships give us access to the best resources and tools for the job.

Get software testing training resources:

Risk-based testing is an approach to testing that optimizes the reduction of risk to the quality of the product, allocating test effort and prioritizing test execution based on risk. It not only reduces the residual level of quality risk but also informs stakeholders of the residual level of risk, starting in the initial stages of a project. It involves the identification of product quality risks and the use of risk levels to guide the test process.


Resources for “Risk Based”

Articles

More and more projects involve more integration of custom developed or commercial-off-the-shelf (COTS) components, rather than in-house development or enhancement of software. In effect, these two approaches constitute direct or indirect outsourcing of some or all of the development work for a system, respectively. While some project managers see such outsourcing of development as reducing the overall risk, each integrated component can bring with it significantly increased risks to system quality. Read this software testing article to learn about the factors that lead to these risks ,and strategies you can use to manage them.

Continue reading →

Since it is not possible to test everything, it is necessary to pick a subset of the overall set of tests to be run. Read this article to discover how quality risks analysis can help one focus the test effort.

Continue reading →

A Case Study in Successful Risk-Based Testing at CA

By Rex Black, Peter Nash and Ken Young

This article presents a case study of a risk-based testing pilot project at CA, the world's leading independent IT management software company. The development team chosen for this pilot is responsible for a widely-used mainframe software product called CA SYSVIEWR Performance Management, an intuitive tool for proactive management and real-time monitoring of z/OS environments.  By analyzing a vast array of performance metrics, CA SYSVIEW can help organizations identify and resolve problems quickly.

CA piloted risk-based testing as part of our larger effort to ensure the quality of the solutions we deliver.  The pilot consisted of six main activities:

  • Training key stakeholders on risk-based testing
  • Holding a quality risk analysis session
  • Analyzing and refining the quality risk analysis
  • Aligning the testing with the quality risks
  • Guiding the testing based on risks
  • Assessing benefits and lessons

This article addresses each of these areas - as well as some of the broader issues associated with risk-based testing.  Click here to read the version of this software testing article as published in Better Software Testing.

Continue reading →

Rex Black’s pioneering Managing the Testing Process was both the first test management book and the first to discuss risk-based testing.  In this software testing article, Rex explains:

  • The benefits of risk-based testing.  
  • Why adding risk analysis to the test team’s responsibilities actually reduces their workload.
  • The importance of stakeholder participation.
  • Common mistakes that can occur in risk-based testing.

Rex illustrates these points, not through hypothetical discussion, but by examining a case study where RBCS helped a client launch risk-based testing.  Read this article to learn how to analyze risks to quality, and use that analysis to be a smarter test professional.

Continue reading →

Analytical risk based testing offers a number of benefits to test teams and organizations that use this strategy. One of those benefits is the opportunity to make risk-aware release decisions.  However, this benefit requires risk based test results reporting, which many organizations have found particularly challenging.  This article describes the basics of risk based testing results reporting, then shows how Rex Black (of RBCS) and Nagata Atsushi (of Sony) developed and implemented new and ground-breaking ways to report test results based on risk.

Testing can be thought of as (one) way to reduce the risks to system quality prior to release. Quality risks typically include possible situations like slow system response to use input, incorrect calculations, corruption of customer data, and difficulty in understanding system interfaces. All testing strategies, competently executed, will reduce quality risks.  However, analytical risk based testing, a strategy that allocates testing effort and sequences test execution based on risk, minimizes the level of residual quality risk for any given amount of testing effort.

There are various techniques for risk based testing, including highly formal techniques like Failure Mode and Effect Analysis (FMEA). Most organizations find this technique too difficult to implement, so RBCS typically recommends and helps clients to implement a technique called Pragmatic Risk Analysis and Management (PRAM).  You can find a case study of PRAM implementation at another large company, CA here. While this article describes the implementation of the technique for projects following a sequential lifecycle, a similar approach has been implemented by organizations using Agile and iterative lifecycle models.

Continue reading →

Analytical risk based testing offers a number of benefits to test teams and organizations that use this strategy.  One of those benefits is the opportunity to make risk-aware release decisions.  However, this benefit requires risk based test results reporting, which many organizations have found particularly challenging.  This article describes the basics of risk based testing results reporting, then shows how Rex Black (of RBCS) and Nagata Atsushi (of Sony) developed and implemented new and ground-breaking ways to report test results based on risk.

Testing can be thought of as (one) way to reduce the risks to system quality prior to release.  Quality risks typically include possible situations like slow system response to use input, incorrect calculations, corruption of customer data, and difficulty in understanding system interfaces.  All testing strategies, competently executed, will reduce quality risks.  However, analytical risk based testing, a strategy that allocates testing effort and sequences test execution based on risk, minimizes the level of residual quality risk for any given amount of testing effort.

There are various techniques for risk based testing, including highly formal techniques like Failure Mode and Effect Analysis (FMEA).  Most organizations find this technique too difficult to implement, so RBCS typically recommends ­and helps clients to implement­ a technique called Pragmatic Risk Analysis and Management (PRAM).  You can find a case study of PRAM implementation at another large company, CA, here. While this article describes the implementation of the technique for projects following a sequential lifecycle, a similar approach has been implemented by organizations using Agile and iterative lifecycle models.

This article was originally published in Software Test and Quality Assurance www.softwaretestpro.com in their December 2010 edition.

Continue reading →

In the Pragmatic Risk Analysis and Management process described in books such as Managing the Testing Process, Pragmatic Software Testing, and Advanced Software Testing: Volume 2, I define the following extents of testing, in decreasing order of thoroughness:

  • Extensive
  • Broad
  • Cursory
  • Opportunity
  • Report bugs only
  • None

Risk based testing does not prescribe specific test design techniques to mitigate quality risks based on the level of risk, as the selection of test design technique for a given risk item is subject to many factors. These factors include the suspected defects (what Boris Beizer called the “bug hypothesis”), the technology of the system under test, and so forth. However, risk based testing does give guidance in terms of the level of test design, implementation, and execution effort to expend, and that does influence the selection of test design techniques. This sidebar provides heuristic guides to help test managers and engineers select appropriate test techniques based on the extent of testing indicated for a risk item by the quality risk analysis process. These guides apply to testing during system and system integration testing by independent test teams.

Continue reading →

More and more projects involve more integration of custom developed or commercial-off-the-shelf (COTS) components, rather than in-house development or enhancement of software.  In effect, these two approaches constitute direct or indirect outsourcing of some or all of the development work for a system, respectively. 

While some project managers see such outsourcing of development as reducing the overall risk, each integrated component can bring with it significantly increased risks to system quality.   In this article, I’ll explain the factors that lead to these risks, and then strategies you can use to manage them. 

I’ll illustrate the factors and the strategies with a hypothetical project.  In this project, assume you are the project manager for a bank that is creating a Web application that allows homeowners to apply for a home equity loan on the Internet.  You have two component vendors.  You buy a COTS database management system from one vendor.  You will hire an outsourced custom development organization to develop the Web pages, the business logic on the servers, and the database schemas and commands to manage the data.  Let’s see how you can recognize the factors that create quality risks, and the strategies you can use to manage those risks.

Continue reading →

Quality Risk Analysis

By Rex Black

Testing any real-world system is potentially an infinite task. Of this infinite set of possible tests, test managers need to focus on the most significant risks to system quality. These are the potential failures that are likely to occur in real-world use or would cost a lot if they did occur. This article describes practical ways to analyze the risks to system quality, providing guidance along the way to achieving effective and efficient testing. 

Continue reading →

Risk Perception

By Erik Simmon

Risk is a highly studied subject:

  • 249 books found on an Amazingly Large Bookstore’s Website
  • Hundreds of papers, conference proceedings, tutorials, classes, and book chapters

But, few of us understand the human side of risk

Continue reading →

If you are a software developer, software development manager, or software quality assurance staff member, you probably know that developing secure software is no longer simply desirable—it’s completely essential. 

Continue reading →

Recently, we worked on a high-risk, high-visibility system where performance testing (“Let’s just make sure it handles the load”) was the last item on the agenda. As luck would have it, the system didn’t handle the load, and very long days and nights ensued. Delivery was late, several serious disasters were narrowly averted, and large costs were incurred.

Continue reading →

Before we can build a high-fidelity test system, we have to understand what quality means to our customers. Test professionals can avail themselves of three powerful techniques for analyzing risks to system quality. Targeting our testing investment by increasing effort for those areas most at risk results in the highest return on investment. 

Continue reading →

Seven Steps to Reducing Software Security Risks

By Rex Black, President, RBCS, Inc.

If you are a software tester, programmer, or manager, you probably know that developing secure software is no longer simply desirable—it’s completely essential.  

Some might assume that most security problems arise from the operating system or networking layers, well below the application code they are working on.  However, figures for Web-based applications show that over three-quarters of security exploits arose from applications. 

So, you know you need secure code, but how to get there? What are your security risks?  What security failures and bugs do you have?  What do these security risks, failures, and bugs mean?  How can you reduce security risk in a way that doesn’t create new problems?  How do you monitor my progress over time?  This article will outline seven steps that will allow you to answer these and other questions as you improve your software’s security.

Continue reading →

Webinars

Podcast Episodes

What is Risk 12/15/2009


Length: 0h 2m 19s

The first installment in a special yearlong series on risk based testing that will share Rex Black's insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Releasing Software without a Blindfold On 12/21/2009


Length: 0h 1m 55s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black's insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Risk Based Test Triage 1/18/2010


Length: 0h 1m 50s

The next installment in a special yearlong series on risk based testing that shares Rex Black's insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

How Risks Can Interact 02/15/2010


Length: 0h 2m 0s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Quality Risk Analysis by Brainstorming 3/10/2010


Length: 0h 3m 4s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Risk Identification by Brainstorming 4/13/2010


Length: 0h 2m 26s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Risk Based Testing 4/16/2010


Length: 1h 29m 19s

Over the last fifteen years, leading professionals working in software and system testing have learned how to apply the powerful techniques of risk analysis and risk management to their projects. Rex Black’s pioneering Managing the Testing Process was both the first test management book and the first to discuss risk based testing. In this Webinar, Rex will explain: the benefits of risk based testing; how risk analysis makes teams more efficient; the how, why, and who of stakeholder involvement;, and common mistakes to avoid. Re x will illustrate these points by examining a case study where RBCS helped a client launch risk-based testing. In this Webinar, you’ll learn how to analyze risks to quality, and use that analysis to be a smarter test professional.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Risk Irrationality 5/3/2010


Length: 0h 1m 49s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black's insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black's insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Software Testing: Why PRAM Is Enough 7/9/2010


Length: 0h 3m 41s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Software Testing: FMEA versus PRAM 8/5/2010


Length: 0h 5m 54s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Software Testing: Managing Project Risks 9/13/2010


Length: 0h 2m 43s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Software Testing: Risk Based Test Design 9/22/2010


Length: 0h 2m 18s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Software Testing: Reporting Residual Risk 10/04/2010


Length: 0h 3m 58s

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The next installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

The final installment in a special yearlong series on risk based testing that shares Rex Black’s insights based on over 15 years of using, improving, and teaching risk based testing.

Listen now →

Mark Twain once wrote, “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.” In logic, that thing you know that just ain’t so is called a fallacy. Fallacies abound in software testing, and risk-based testing especially suffers from a number of misconceptions. Are you laboring under such delusions? If so, what damage is that doing to your ability to implement the most effective and efficient way to manage the risk to the quality of your software and systems? In this free webinar, Rex will identify and dispel these fallacies, and take your questions about risk-based testing fallacies that are affecting you.

Listen now →

Webinar: Two Bug Metrics, Millions in Process Improvement


Length: 1h 20m 50s

When we do assessments, we always try to look at process metrics. In most cases, we can find millions of dollars in process improvement opportunities. In this webinar, Rex will show you how two very simple bug metrics, calculated using only two simple facts for each bug report using simple, free spreadsheets you can get from our website, can reveal millions and millions of dollars in potential process improvements. All the more reason to track those bugs! To paraphrase Timothy Leary: Tune in, download, and drop software co

Listen now →

Training

ISTQB Virtual Advanced Security Tester Boot Camp

The Advanced Security Tester Boot Camp course, created by Rex Black, past President of the International Software Testing Qualifications Board (ISTQB), past President of the American Software Testing Qualifications Board (ASTQB) and co-author of a number of International Software Testing Qualifications Board syllabi, is ideal for testers and test teams preparing for certification in a short timeframe with time and money constraints.

View details →


Copyright ® 2017 Rex Black Consulting Services.
All Rights Reserved.
ISTQB Logo ASTQB Logo IREB Logo PMI Logo IIBA Logo ISTQB Logo
PMI is a registered mark of the Project Management Institute, Inc.