Resources

Our notable partnerships give us access to the best resources and tools for the job.

Get software testing training resources:

Test management is the planning, estimating, monitoring and control of test activities, typically carried out by a test manager.


Resources for “Management”

Articles

More and more projects involve more integration of custom developed or commercial-off-the-shelf (COTS) components, rather than in-house development or enhancement of software. In effect, these two approaches constitute direct or indirect outsourcing of some or all of the development work for a system, respectively. While some project managers see such outsourcing of development as reducing the overall risk, each integrated component can bring with it significantly increased risks to system quality. Read this software testing article to learn about the factors that lead to these risks ,and strategies you can use to manage them.

Continue reading →

Empirix's QAZone with Rex Black

By Marina Gil Santamaria

From certification to automation, expert thoughts on where the testing industry is and where it's headed.

Continue reading →

Recently, we worked on a high-risk, high-visibility system where performance testing ("Let's just make sure it handles the load") was the last item on the agenda. As luck would have it, the system didn't handle the load, and very long days and nights ensued. Why it does have to be this way... Read this article about Risk Mitigation to ensure this doesn’t happen to you.

Continue reading →

"Do more with less. Work smarter not harder. Same coverage, fewer testers." If you're like a lot of testers and test managers, you'll be hearing statements like those a lot in 2009, since we appear headed for another tight economic period. If you need a way to demonstrate quick, measurable efficiency gains in your test operation, read this  short article to learn four great ideas that will help you improve your software test efficiency. 

Continue reading →

A Case Study in Successful Risk-Based Testing at CA

By Rex Black, Peter Nash and Ken Young

This article presents a case study of a risk-based testing pilot project at CA, the world's leading independent IT management software company. The development team chosen for this pilot is responsible for a widely-used mainframe software product called CA SYSVIEWR Performance Management, an intuitive tool for proactive management and real-time monitoring of z/OS environments.  By analyzing a vast array of performance metrics, CA SYSVIEW can help organizations identify and resolve problems quickly.

CA piloted risk-based testing as part of our larger effort to ensure the quality of the solutions we deliver.  The pilot consisted of six main activities:

  • Training key stakeholders on risk-based testing
  • Holding a quality risk analysis session
  • Analyzing and refining the quality risk analysis
  • Aligning the testing with the quality risks
  • Guiding the testing based on risks
  • Assessing benefits and lessons

This article addresses each of these areas - as well as some of the broader issues associated with risk-based testing.  Click here to read the version of this software testing article as published in Better Software Testing.

Continue reading →

When I wrote my book Critical Testing Processes in the early 2000s, I started with the premise that some test processes are critical, some are not. I designed this lightweight framework for test process improvement in order to focus the test team and test manager on a few test areas that they simply must do properly. This contrasts with the more expansive and complex models inherent in TPI and TMM.  In addition, the Critical Testing Processes (CTP) framework eschews the prescriptive elements of TMM and TPI since it does not impose an arbitrary, staged maturity model.

What’s the problem with prescriptive models?  In my consulting work, I have found that businesses want to make improvements based on the business value of the improvement and the organizational pain that improvement will alleviate. A simplistic maturity rating might lead a business to make improvements in parts of the overall software process or test process that are actually less problematic or less important than other parts of the process simply because the model listed them in order.

CTP is a non-prescriptive process model. It describes the important software processes and what should happen in them, but it doesn’t put them in any order of improvement. This makes CTP a very flexible model. It allows you to identify and deal with specific challenges to your test processes. It identifies various attributes of good processes, both quantitative and qualitative. It allows you to use business value and organizational pain to select the order and importance of improvements. It is also adaptable to all software development lifecycle models.

Continue reading →

Analytical risk based testing offers a number of benefits to test teams and organizations that use this strategy. One of those benefits is the opportunity to make risk-aware release decisions.  However, this benefit requires risk based test results reporting, which many organizations have found particularly challenging.  This article describes the basics of risk based testing results reporting, then shows how Rex Black (of RBCS) and Nagata Atsushi (of Sony) developed and implemented new and ground-breaking ways to report test results based on risk.

Testing can be thought of as (one) way to reduce the risks to system quality prior to release. Quality risks typically include possible situations like slow system response to use input, incorrect calculations, corruption of customer data, and difficulty in understanding system interfaces. All testing strategies, competently executed, will reduce quality risks.  However, analytical risk based testing, a strategy that allocates testing effort and sequences test execution based on risk, minimizes the level of residual quality risk for any given amount of testing effort.

There are various techniques for risk based testing, including highly formal techniques like Failure Mode and Effect Analysis (FMEA). Most organizations find this technique too difficult to implement, so RBCS typically recommends and helps clients to implement a technique called Pragmatic Risk Analysis and Management (PRAM).  You can find a case study of PRAM implementation at another large company, CA here. While this article describes the implementation of the technique for projects following a sequential lifecycle, a similar approach has been implemented by organizations using Agile and iterative lifecycle models.

Continue reading →

Analytical risk based testing offers a number of benefits to test teams and organizations that use this strategy.  One of those benefits is the opportunity to make risk-aware release decisions.  However, this benefit requires risk based test results reporting, which many organizations have found particularly challenging.  This article describes the basics of risk based testing results reporting, then shows how Rex Black (of RBCS) and Nagata Atsushi (of Sony) developed and implemented new and ground-breaking ways to report test results based on risk.

Testing can be thought of as (one) way to reduce the risks to system quality prior to release.  Quality risks typically include possible situations like slow system response to use input, incorrect calculations, corruption of customer data, and difficulty in understanding system interfaces.  All testing strategies, competently executed, will reduce quality risks.  However, analytical risk based testing, a strategy that allocates testing effort and sequences test execution based on risk, minimizes the level of residual quality risk for any given amount of testing effort.

There are various techniques for risk based testing, including highly formal techniques like Failure Mode and Effect Analysis (FMEA).  Most organizations find this technique too difficult to implement, so RBCS typically recommends ­and helps clients to implement­ a technique called Pragmatic Risk Analysis and Management (PRAM).  You can find a case study of PRAM implementation at another large company, CA, here. While this article describes the implementation of the technique for projects following a sequential lifecycle, a similar approach has been implemented by organizations using Agile and iterative lifecycle models.

This article was originally published in Software Test and Quality Assurance www.softwaretestpro.com in their December 2010 edition.

Continue reading →

Organizing Manual Testing on a Budget

By Capers Jones, Vice President and Chief Technology Officer - Namcook Analytics LLC

RBCS is pleased to feature a special guest author for our newsletter article, Capers Jones. Capers Jones is, of course, a long-standing force for improving the software engineering industry, and has published a number of books that I consider essential reading for software professionals who seek to truly understand, through data and facts, what happens on software projects. Recently, he published an important book on software quality, The Economics of Software Quality. So, I asked Capers if he'd be willing to contribute a guest article, and he graciously agreed. This article, on software quality today and tomorrow, gives us a sobering view of our current situation, but also provides clear direction on what we need to do to get better. The good news is that we already have many of the tools we need to improve software quality. -- Rex Black 

In 2012 large software projects are hazardous business undertakings. More than half of software projects larger than 10,000 function points (about 1,000,000 lines of code) are either cancelled or run late by more than a year. 

When examining troubled software projects, it always happens that the main reason for delay or termination is due to excessive volumes of serious defects. Conversely, large software projects that are successful are always characterized by excellence in both defect prevention and defect removal. It can be concluded that achieving state of the art levels of software quality control is the most important single objective of software process improvements.

Quality control is on the critical path for advancing software engineering from its current status as a skilled craft to become a true profession.

Continue reading →

This article is excerpted from Chapter 3 of Rex Black’s book Managing the Testing Process, 3e.

A number of RBCS clients find that obtaining good test data poses many challenges. For any large-scale system, testers usually cannot create sufficient and sufficiently diverse test data by hand; i.e., one record at a time. While data-generation tools exist and can create almost unlimited amounts of data, the data so generated often do not exhibit the same diversity and distribution of values as production data. For these reasons, many of our clients consider production data ideal for testing, particularly for systems where large sets of records have accumulated over years of use with various revisions of the systems currently in use, and systems previously in use. 

However, to use production data, we must preserve privacy. Production data often contains personal data about individuals which must be handled securely. However, requiring secure data handling during testing activities imposes undesirable inefficiencies and constraints. Therefore, many organizations want to anonymize (scramble) the production data prior to using it for testing.

This anonymization process leads to the next set of challenges, though. The anonymization process must occur securely, in the sense that it is not reversible should the data fall into the wrong hands. For example, simply substituting the next digit or the next letter in sequence would be obvious to anyone­ it doesn’t take long to deduce that "Kpio Cspxo" is actually "John Brown" ­which makes the de-anonymization process trivial. 

Continue reading →

Defining Test Mission, Policy, and Metrics of Success

By Rex Black and Debbie Friedenberg

The International Software Testing Qualifications Board (ISTQB) program defines testing expansively. Static testing (via reviews and static analysis) is included as well as all levels of dynamic testing, from unit testing through to the various forms of acceptance testing. The test process is defined to include all necessary activities for these types of testing; planning; analysis, design, and implementation; execution, monitoring, and results reporting; and closure.

Continue reading →

An article featuring an interview with Rex Black was recently published in “Tester’s Life” magazine.   Click below to read the interview in English or, to see the original source article in it's entirety in Russian, visit www.testers-life.ru.  

Continue reading →

When I do assessments for clients, I talk to a lot of people, both inside and outside the testing group. In the opening moments of each interview, I try to engage in a friendly exchange, where I break the ice between the interviewee and myself. Not only is it more pleasant to have a friendly conversation than a tense one, but people are more open and honest with someone with whom they have some kind of positive relationship, compared to a complete stranger—or someone they see as hostile, cold, or inscrutable. Most of the time, I succeed, and I get to spend an interesting hour or so with someone who gives me the benefit of their insights and opinions.

The same is true, on a much larger and longer scale, for test managers. Testing is a matter of providing useful services to stakeholders. If those stakeholders have a good relationship with you and the other test managers in your test group, information will flow more smoothly in both directions. The job of the test group will become easier because it has better access to information it needs. The test group will also become more valuable because the information the group produces will flow more smoothly to the recipients of that information. It’s just human nature: We listen to and value the communications we receive from people we are comfortable with, and we are happy to reciprocate that flow of information.

It’s not that you must be a personal friend to every stakeholder with whom you work, but a good professional relationship with those stakeholders is a major factor in the success of a test manager. How well you and the other managers in the test group initiate, cultivate, and sustain these relationships will strongly influence the flow of information, as well as the support, you obtain from your colleagues.

A relationship is necessarily a two-way affair. You and the test group can’t be the only beneficiaries from a relationship, at least not a good one. Once, a person with whom I worked on a project described the CEO of one vendor as follows: “Every time I meet with that guy, I want to take a shower afterward,” meaning that he felt soiled just by being in the same room. Later in the project, when my colleague legitimately but accidentally came into possession of a memo that was certainly not in the vendor’s interests to disclose to its client, my colleague felt no compunction about copying the document before returning it in a way that did not disclose that he had seen it. The relationship had become two-way, but not in a good way.
As a contrast, I had an excellent relationship with this same vendor’s test manager. Across a significant cultural difference—the same difference my colleague and the CEO had not bridged—he and I forged a relationship of honesty and trust. I felt I could tell him the truth about what was happening on my side of the project, and he felt the same. We shared information to advance our mutual goals of a successful project and high-quality deliverable while at the same time respecting the limits on communication imposed by our different positions in terms of who our employers were. Even when the relationship between the two companies became testy, he and I were always able to communicate as friends with a good relationship of mutual respect.

I note that this anecdote does not represent an isolated incident but rather a truth that has become plain to me throughout my career in testing. The successful test manager, perhaps more than any other managers in the software business, must cultivate strong relationships with stakeholders, continuously reinforce those relationships with mutual benefits, and maintain the relationships through good times and bad. In the next few subsections, let’s look more closely at how.

Continue reading →

Back in the early days of computing, there was no such specific, separately identifiable activity known as “testing.” Developers debugged their code, and that was usually intertwined with some unit testing task. That didn’t work. My first job was as a programmer, and where I worked, this was exactly how we approached quality assurance. It was a quality disaster.

This approach by itself still doesn’t work. It does not work for those throwback, Neanderthal organizations that rely on this approach entirely. It does not work for those cutting-edge companies that think some fancy language or process or tool has solved the software quality problem at last. It does not work for anyone, unless we define “work” as “shift most costs of poor quality onto end users who are too stupid to know better or trapped in a monopolistic market without any real choice.” And those organizations that rely on stupid users or a monopoly position had better hope they are right.

With the widespread advent of independent test teams in the late 1980s and early 1990s, we saw improvements. I was working in an independent test lab in the late 1980s and as a test manager in the early 1990s, and we made great strides. However, we also saw the emergence of a new dysfunction, the “hurl it over the wall to the test guys and hold them responsible for delivered quality” mistake. Every now and then, we work with organizations that still suffer from this problem.

Let’s be clear. When quality matters—and shouldn’t it always—everyone must play a role. Good testing involves a series of quality assurance filters. Each team in the organization typically participates in and owns one or more of these filters.

Continue reading →

Bug Reporting Process

By Rex Black

Last time, I talked about an internal test process, managing test execution. This is a process that is only indirectly visible to the rest of the development team, in that, as long as you get through all your planned tests, effectively respond to change, and report your findings intelligibly, then for all most people care your testing could occur through voodoo, augury, and a Ouija board.

Continue reading →

Why Care About Testing Processes?

  • Bugs are expensive
    US economy loses $60B yearly due to bugs, $20B of which could be saved through better testing
  • Bugs drive away customers
    Handwriting recognition bugs in Newton cost Apple the person digital assistant (PDA) market
  • Bugs sometimes kill people
    Therac-25 overdoses, Patriot clock overflow
  • Inadequate testing endangers the project
    Bad testing processes can increase the risk of project delay or cancellation by 25 to 300% 

Continue reading →

As a test professional with almost two decades in the computer field, I’m pleased to see a new publication devoted to our field, the Journal of Software Testing Professionals, and honored to serve as a Contributing Editor. As my first task in this role, I have been asked to write and collaborate with other software testing professionals to produce a series of articles addressing what we consider critical processes that the test professional must master in order for her to succeed. In each of these articles, we’ll examine one critical process closely, offering practical advice on what works and caveats on what can cause problems

Continue reading →

So, you are responsible for managing a computer hardware or software test project? Congratulations! Maybe you’ve just moved up from test engineering or moved over from another part of the development team, or maybe you’ve been doing test projects for a while. Whether you are a test manager, a development manager, a technical or project leader, or an individual contributor with some level of responsibility for your company’s test and quality assurance program, you’re probably looking for some ideas on how to manage the unique beast that is a test project.

Continue reading →

There are two measures that have a strong influence on the outcomes of software projects: 1) Defect potentials; 2) Defect removal efficiency.

The term “defect potentials” refers to the total quantity of bugs or defects that will be found in five software artifacts: requirements, design, code, documents, and “bad fixes” or secondary defects. 

Continue reading →

Mission Made Possible

By Rex Black and Greg Kubaczkowski

Your mission should you choose to accept it: A client needs to implement an integrated, automated unit, component, and integration testing process. The system development teams are spread across 3 continents. They use multiple platforms. Their system architecture is interdependent and complex. Some tests need to be run on an application server. Both static and dynamic testing is required. The process you design must be simple to use, not an additional burden for busy developers. It should be automated and easily integrated. By the way, only 3 designers have been assigned to this mission. You have 6 weeks. Good luck. This message will self-destruct in 20 seconds… 

Continue reading →

In the last issue, I dealt with a collaborative test process, getting test releases into the test lab. In this article, let’s look instead at an internal test process, managing the execution of tests against a test release. One can think of the test manager’s role during the test execution process as one of managing a search for the unexpected. While all managers are expected to manage unexpected events within their areas of responsibility, the test manager is the only manager in the software development organization whose area of responsibility is searching for the unexpected. This search takes the form of running a set of tests. How does a test team take these tests, run them against an installed system, and capture information about the system under test, the test cases, and the test execution process itself? 

Continue reading →

Project managers must develop quality systems that provide the needed features in a timely and cost-effective fashion. Testing systems prior to release is necessary to assess their quality, but what’s the return on that testing investment? This article describes four quantifiable metrics for testing ROI

Continue reading →

Recently, we worked on a high-risk, high-visibility system where performance testing (“Let’s just make sure it handles the load”) was the last item on the agenda. As luck would have it, the system didn’t handle the load, and very long days and nights ensued. Delivery was late, several serious disasters were narrowly averted, and large costs were incurred.

Continue reading →

Most software test teams exist to assess the software’s readiness prior to release. To achieve this goal, two primary tactics are used:

  1. Execute test cases or scenarios that are likely to find errors, resemble actual usage, or both.
  2. Report the test results, the defects found, and defects fixed, which, collectively, make up the test status and reflect the quality of the software.

For the test manager, the first task category primarily involves managing inward: assembling the test team and test resources, designing a solid test system, implementing the test system, and running the tests in an intelligent, logical fashion. The second area of responsibility involves upward and outward management. Upward management includes how you communicate with your managers, other senior managers, and executive staff. Outward management includes communication with management peers in development, configuration/release management, sales, marketing, finance, operations, technical support, and so on. As a test manager, your effectiveness in reporting test status has a lot to do with both your real and perceived effectiveness in your position. To put it another way, your management superiors and peers measure your managerial competence as much by how well you report your results as by how you obtain your results. 

Continue reading →

Realizing a solid return on your testing investment requires smart selection of tests. Cost of quality analysis tells us that it’s cheaper to find and fix bugs before the customers do, but, to keep bugs away from customers, we have to find the ones that matter. Doing so requires that we understand how the customers will use the system. 

Continue reading →

Automated test tools are powerful aids to improving the return on the testing investment when used wisely. Some tests inherently require an automated approach to be effective, but others must be manual. In addition, automated testing projects that fail are expensive and politically dangerous. How can we recognize whether to automate a test or run it manually, and how much money should we spend on a test? 

Continue reading →

Pervasive testing means getting the right people working together through the right processes at the right time for high-ROI testing. Pervasive testing is the final piece of the test investment puzzle, and just as important as all the others. Through pervasive testing, all the ideas we’ve explored so far come together.

Continue reading →

Subtle but catastrophic bugs, such as those that cause server crashes and database record-lock race conditions, often only reveal themselves during performance, stress, volume, data quality, and reliability testing. Such testing is most effectively performed in test environments— hardware, software, network, and release configurations—that mimic as nearly as possible the field environment, because test results in less-complex settings often do not extrapolate due to the non-linearity of software. In complex settings, such as Web and e-commerce server and database farms, managing these lab configurations can be quite challenging. This paper presents a basic Access database, designed using the Entity-Relationship technique, that will allow the Test Manager to plan, configure, and maintain this test environment through the test project. 

Continue reading →

Information appliances, which provide simplified, easy access to specific information such as e-mail and Web sites, promise to bring the benefits of computing to a wide customer base, including some computer-averse people who have hitherto avoided buying a computer. Internet appliances are evolving from personal computers, game stations, digital mobile phones, and server technologies. While this allows us to apply well-known quality assurance techniques, including testing techniques, the software quality professional must remember that the risks to product quality are different; the quality bar is higher, especially in terms of usability, robustness, and harmonizing the appliance with the dynamic Internet. Customers will assess the quality of information appliances by the degree to which the appliance reliably, quickly, transparently, and intuitively provides them with access to the desired information, and we expect them to be much less understanding of glitches than the current PC user. Information appliances are gaining wide acceptance—millions will hit the market in the next few years—so many of us who practice software quality professions will spend time working on projects to develop them. Indeed, we expect that information appliances will present tremendous opportunities to those who seek to bring quality to software in the new millennium. This paper presents the test team’s findings on one such project.

Continue reading →

In the Test Manager’s perfect world, her team would be staffed entirely by expert software engineering professionals. These test engineers could converse intelligently one minute with development engineers about code coverage and memory leaks; the next minute talk with technical support agents about the customer’s experience of quality; and one minute later speak to marketeers about trade-offs between features, schedule, budget, and quality in the upcoming release. Such engineers would spend their time programming unit and integration test stubs and scaffolds for structural testing, scripting automated behavioral tests at the GUI level, and creating load generators and performance probes. Manual testing would be used sparingly to fill the gaps in such tests, for example by producing interesting error conditions and checking the system’s response. The Test Manager would have ample staffing budget to hire such peer-level test engineers, and would have early involvement in the development effort. He could focus on verifying quality risk coverage using techniques like failure mode and effect analysis, analyzing and reporting defect management metrics, and setting up career development plans for his engineers.

Continue reading →

Suppose you went to a restaurant for dinner, sat down, and told the waiter, "Bring me dinner and a drink." You didn't provide any further details, though you had something specific in mind. What are the chances that you'll get the dinner and drink you expected? While no one would ever do this in a restaurant, it happens sometimes on projects that involve third parties.

If we have certain expectations and requirements for an engagement with a third party, those should be defined and clearly communicated between the parties. The best practice is to have that definition and communication before the project starts and to put the agreed-upon terms into the contract. If the third party is delivering software, then these requirements should include quality targets, including measurements of those targets. The measurements should be objective and not subject to distortions.

In addition to defining the requirements, the point at which those requirements must be met should be defined. This can be done by defining entry and exit criteria that establish quality gates for deliverables. Because these quality gates will control the start and end of project phases, they should be synchronized with the phases of the project and aligned with project schedule milestones. 

Continue reading →

Project Retrospective

By Rex Black

An excerpt from The Expert Test Manager: Guide to the ISTQB Expert Level Certification book by Rex Black, Jim Rommens and Leo Van Der Aalst due to be published by Rocky Nook. All material is provisional and may be subject to change
 
As a colleague told me once, a good motto for software teams is: "Make interesting new mistakes." His explanation was, since you are human, you'll make mistakes. But you should make interesting ones, ones you can learn from, and you should only make a mistake once.  How do you ensure that you make only interesting new mistakes? By learning from each mistake that you make. How to you learn from each mistake? In this short article, you'll read about a proven technique for learning from mistakes, retrospectives. Useful in both Agile and traditional lifecycles, this simple technique can make you and your colleagues a process improvement machine!
 

Continue reading →

"Back in 2010, at the launch of Core Magazine, http://www.coremag.eu/, I wrote a series of columns to welcome people to the magazine. As a sort of Throw-Back-December, here they are, as they appeared in the original magazine issues. I hope you enjoy them."
-Rex Black
 
Greetings, and welcome to my quarterly column on software testing best practices.  When I was asked to write this column, I had to choose the approach, the theme.  The writers' aphorism says, "Write what you know." So, what do I know?
 
Well, if you know me and my consulting company, RBCS, you know that we spend time with clients around the world, in every possible industry, helping people improve their testing with training or consulting services, or doing testing for them with our outsourcing services.  Our work gives me insights into what goes on, the actual day-to-day practice of software testing.
 

Continue reading →

Seven Steps to Reducing Software Security Risks

By Rex Black, President, RBCS, Inc.

If you are a software tester, programmer, or manager, you probably know that developing secure software is no longer simply desirable—it’s completely essential.  

Some might assume that most security problems arise from the operating system or networking layers, well below the application code they are working on.  However, figures for Web-based applications show that over three-quarters of security exploits arose from applications. 

So, you know you need secure code, but how to get there? What are your security risks?  What security failures and bugs do you have?  What do these security risks, failures, and bugs mean?  How can you reduce security risk in a way that doesn’t create new problems?  How do you monitor my progress over time?  This article will outline seven steps that will allow you to answer these and other questions as you improve your software’s security.

Continue reading →

This is an excerpt from my book, Expert Test Manager, written with James Rommens and Leo van der Aalst. I hope it helps you think more clearly about the test strategies you use.

A test policy contains the mission and objectives of testing along with metrics and goals associated with the effectiveness, efficiency, and satisfaction with which we achieve those objectives. In short, the policy defines why we test. While it might also include some high-level description of the fundamental test process, in general the test policy does not talk about how we test.

The document that describes how we test is the test strategy. In the test strategy, the test group explains how the test policy will be implemented. This document should be a general description that spans multiple projects. While the test strategy can describe how testing is done for all projects, organizations might choose to have separate documents for various types of projects. For example, an organization might have a sequential lifecycle test strategy, an Agile test strategy, and a maintenance test strategy.

Continue reading →

Test-Driven Development, Acceptance Test-Driven Development, and Behaviour-Driven Development

By by Rex Black, Marie Walsh, Gerry Coleman, Bertrand Cornanguer, Istvan Forgacs, Kari Kakkonen, and Jan Sabak

[Note: This is an excerpt from Agile Testing Foundations: An ISTQB Foundation Level Agile Tester Guide, by Rex Black, Marie Walsh, Gerry Coleman, Bertrand Cornanguer, Istvan Forgacs, Kari Kakkonen, and Jan Sabak, published July 2017. Kari Kakkonen wrote this selection. The authors are all members of the ISTQB Working Group that wrote the ISTQB Agile Tester Foundation syllabus.]

The traditional way of developing code is to write the code first, and then test it. Some of the major challenges of this approach are that testing is generally conducted late in the process and it is difficult to achieve adequate test coverage. Test-first practices can help solve these challenges. In this environment, tests are designed first, in a collaboration between business stakeholders, testers, and developers.  Their knowledge of what will be tested helps developers write code that fulfils the tests. A test-first approach allows the team to focus on and clarify the expressed needs through a discussion of how to test the resulting code. Developers can use these tests to guide their development.  Developers, testers, and business stakeholders can use these tests to verify the code once it is developed.

A number of test-first practices have been created for Agile projects, as mentioned in section 2.1 of this book. They tend to be called X Driven Development, where X stands for the driving force for the development. In Test Driven Development (TDD), the driving force is testing. In Acceptance Test-Driven Development (ATDD), it is the acceptance tests that will verify the implemented user story. In Behaviour-Driven Development (BDD), it is the behaviour of the software that the user will experience. Common to all these approaches is that the tests are written before the code is developed, i.e., they are test-first approaches. The approaches are usually better known by their acronyms. This subsection describes these test-first approaches and information on how to apply them is contained in section 3.3.

Test-Driven Development was the first of these approaches to appear. It was introduced as one of the practices within Extreme Programming (XP) back in 1990. It has been practiced for two decades and has been adopted by many software developers, in Agile and traditional projects. However, it is also a good example of an Agile practice that is not used in all projects. One limitation with TDD is that if the developer misunderstands what the software is to do, the unit tests will also include the same misunderstandings, giving passing results even though the software is not working properly. There is some controversy over whether TDD delivers the benefits it promises. Some, such as Jim Coplien, even suggest that unit testing is mostly waste.

TDD is mostly for unit testing by developers.  Agile teams soon came up with the question: What if we could have a way to get the benefits of test-first development for acceptance tests and higher level testing in general? And thus Acceptance Test-Driven Development was born.  (There are also other names for similar higher-level test-first methods; for example, Specification by Example (SBE) from Gojko Adzic.) Later, Dan North wanted to emphasize the behaviours from a business perspective, leading him to give his technique the name Behaviour-Driven Development. ATDD and BDD are in practice very similar concepts. 

Let’s look at these three test-first techniques, TDD, ATDD, and BDD, more closely in the following subsections.

 

Continue reading →

Webinars

In Praise of Specialization

Recorded August 27, 2014

Towards a True Engineering Profession

Recorded February 18, 2014

The Sherpas of Testing Success

Recorded August 14, 2013

Profiles of Failure

Recorded April 24, 2013

Hiring Great Testers

Recorded November 10, 2010

Test Organization Options

Recorded July 1, 2010

Satisfying Test Stakeholders

Recorded May 20, 2010

Test Reporting for Impact

Recorded November 23, 2009

The Future of Test Management

Recorded October 8, 2009

RBCS Videos: RBCS Services

Recorded August 18, 2009

Ten Worst Things for Testing

Recorded June 9, 2009

Agile Testing Challenges

Recorded April 30, 2009

Podcast Episodes

Webinar: Schools of Testing Debate 4/15/2014


Length: 1h 10m 9s

“Schools of Testing”: Useful Paradigm or Negative Influence? Are there really four different “schools of testing”? Do you have to belong to one? Or are there just different testing strategies, to be selected and blended as appropriate? Has the concept of “schools of testing” promoted clearer thinking about testing, or has this concept actually created a useless schism, sowing conflict and slowing growth of the profession? In this debate, Rex Black argued the negative side of the case, being one of the leading critics of the concept of “schools of testing.” One the other side, you’ll hear one of the originators of the concept, Cem Kaner, argue for its benefits. Love it or hate it, you can hear both sides and make up your own mind.

Listen now →

Webinar: Why Do You Test? 9/8/15


Length: 1h 22m 48s

Why do you test? Have you asked yourself that question before? Try it. Then take the next step: Identify two or three colleagues with different roles and ask them why they think you test. Typically, the lists won’t match up very well, and many times the lists include completely unrealistic objectives like, “Make sure the software works right before we put it into production,” or, “Find all the bugs before we ship.” Instead, in this webinar, Rex will explain a process that will allow you to create a set of clearly defined, achievable, measurable set of objectives. This process will also build consensus on those objectives within your team and across teams. Tune in to this free webinar to step out of the confusion and into clarity on this important topic.

Listen now →

Webinar: The Seven Deadly Sins of Testing


Length: 1h 5m 46s

Many smart, otherwise-capable testers sabotage their own careers by committing one or more of the seven deadly sins of testing.  Are you your own worst enemy?  Come join us to discuss these seven deadly sins with Rex.  You might recognize your own behaviors, or behaviors of others on your test team.  In this updated webinar presentation, Rex gives examples of these behaviors through case studies, and tells you how to stop the behaviors and solve the problems those behaviors have created.  For sinners and non-sinners alike, Rex offers ideas on how to become a testing saint.  

Listen now →

Webinar: Two Bug Metrics, Millions in Process Improvement


Length: 1h 20m 50s

When we do assessments, we always try to look at process metrics. In most cases, we can find millions of dollars in process improvement opportunities. In this webinar, Rex will show you how two very simple bug metrics, calculated using only two simple facts for each bug report using simple, free spreadsheets you can get from our website, can reveal millions and millions of dollars in potential process improvements. All the more reason to track those bugs! To paraphrase Timothy Leary: Tune in, download, and drop software co

Listen now →

Webinar: The Psychopolitics of Test Management 1/11/17


Length: 1h 21m 33s

While much of testing and test management involves rational decision-making, measuring quality and providing testing services to the team, there are realms of the test manager’s job where psychology meets politics to form psychopolitics.   In this webinar, drawn from three decades of industry experience and materials in his best-selling book Managing the Testing Process, 3rd edition, Rex will discuss how psychology and politics can collide to make the test manager’s job…interesting. Join in the discussion after the initial presentation with your own questions and stories about testing psychology, testing politics, and plain ol’ psycho-politics!

Listen now →

Stupid Metrics Tricks and How To Avoid Them


Length: 1h 1m 48s

If you have been in software engineering for a while—or in fact just in the working world in general for a while—you’ve probably seen someone do something stupid with metrics. Such mistakes raise a whole bunch of interesting questions. What are the most common metrics mistakes? Why are they mistakes? Why do people make these mistakes? Are you making these mistakes? Why use metrics at all, when there are so many mistakes? In this talk, Rex will give real-world examples of these mistakes, explain the management and economic theories behind metrics, and help you find ways to implement metrics that aren’t stupid.

Listen now →

For nearly ten years, RBCS has run a highly successful free webinar series.  In 2018, we’re adding the Two Points of View at Two series to our monthly webinar rotation. In each of these sessions, Rex Black will talk with another software luminary about topics of mutual interest, where the two have some different views, and then Rex opens the floor to questions. 

In this inaugural session, Rex is happy to welcome Maaret Pyhäjärvi .  Maaret’s bio describes her as feedback fairy with a day job at F-Secure, where they call her a Lead Quality Engineer. She identifies as empirical technologist, tester and programmer, catalyst for improvement, author and speaker, and community facilitator and conference organizer. You can catch her latest thoughts on her blog at http://visible-quality.blogspot.fi

In this session, Rex and Maaret will discuss tester-developer collaboration and the relationships between testers and developers.  How to approach developers for collaboration? How do testers-developers ratios affect relationships? What about people who move between tester and developer roles? Join Rex and Maaret to hear their thoughts and ask your questions.

Listen now →

One Key Idea: Two Simple, Useful Defect Metrics


Length: 0h 30m 44s

Technical debt is bad.  It’s smart to carefully manage technical debt. Defects are a form of technical debt.  Do you know how to measure how well you are managing defect-related technical debt? In this One Key Idea session, Rex will demonstrate two simple defect metrics, easily extractable from any defect management tool, which can give you useful insights into what’s happening with defect-related technical debt.  In twenty minutes or less, you’ll learn what these metrics can tell you and how you can use them to manage your technical debt better.

Listen now →

The Seven Deadly Sins of Testing


Length: 1h 7m 3s

Many smart, otherwise-capable testers sabotage their own careers by committing one or more of the seven deadly sins of testing.  Are you your own worst enemy?  Come join us to discuss these seven deadly sins with Rex.  You might recognize your own behaviors, or behaviors of others on your test team.  In this webinar presentation, Rex gives examples of these behaviors through case studies, and tells you how to stop the behaviors and solve the problems those behaviors have created.  For sinners and non-sinners alike, Rex offers ideas on how to become a testing saint.

Listen now →

Hiring and managing distributed, international test teams

Phil Lew, President of XBOSoft, an international software testing firm, joins Rex Black to discuss the critical topics of hiring and managing teams of testers who work around the world.  How do distance, culture, and language create challenges, and how have Phil and Rex dealt with those challenges in the past?  You won’t want to miss Phil and Rex’s points of view on these important topics.

Listen now →

What should testing accomplish and how do we recognize success?  How should we approach testing? What specific activities need to be planned for each release, each project, each iteration?  What risks can affect testing, and how can we manage those risks?  How does lifecycle affect these documents? These are critical questions for testers and test managers, yet often they go unanswered or have answers that aren’t fully thought through.  In this webinar, based on years of experience helping testers and test teams optimize their testing processes, Rex will discuss test policies, strategies and plans.  Join this webinar, illustrated with examples throughout, to learn concepts and ideas that you can apply to your testing processes right away.

Listen now →

In this month’s “Two Points of View at Two” session, Rex is happy to welcome Nivia S. Henry. Nivia fundamentally believes that happy people, working in a healthy environment, will do great things. This philosophy has driven her to build a 15+ year career creating and supporting high-performing teams. Her career path has included agile coaching, enterprise agile transformations, product management, and people leadership. Today, Nivia applies her hard-earned experience as an Quality and Web Engineering Manager at Spotify. In this session, she'll discuss Spotify's perspective on Quality and focus on how the organization has organically grown its test automation practice.

Listen now →

One Key Idea: Understanding the ISTQB Career Path


Length: 0h 31m 57s

The ISTQB provides extensive support for the test professional’s career path. There are so many options, in fact, that some people get confused and have questions.  Well, in this One Key Idea session, Rex is here to make the career path clear and answer your questions.  In twenty minutes or less, you’ll understand how the ISTQB career path can support your progress throughout your professional testing life.

Listen now →

Training

ISTQB Virtual Expert Test Management Strategic Test Manager Boot Camp

The Expert Test Management Strategic Test Manager Boot Camp, created by Rex Black, past President of the International Software Testing Qualifications Board (ISTQB), past President of the American Software Testing Qualifications Board (ASTQB) and co-author of the International Software Testing Qualifications Board Expert Level Syllabus Test Management, is ideal for testers and test teams preparing for certification in a short timeframe with time and money constraints. 

View details →

ISTQB Virtual Expert Test Management Operational Test Manager Boot Camp

The Expert Test Management Operational Test Manager Boot Camp, created by Rex Black, past President of the International Software Testing Qualifications Board (ISTQB), past President of the American Software Testing Qualifications Board (ASTQB) and co-author of the International Software Testing Qualifications Board Expert Level Syllabus Test Management, is ideal for testers and test teams preparing for certification in a short timeframe with time and money constraints. 

View details →

ISTQB Virtual Expert Test Management Managing the Test Team Boot Camp

The Expert Test Management Managing the Test Team Module Boot Camp, created by Rex Black, past President of the International Software Testing Qualifications Board (ISTQB), past President of the American Software Testing Qualifications Board (ASTQB) and co-author of the International Software Testing Qualifications Board Expert Level Syllabus Test Management, is ideal for testers and test teams preparing for certification in a short timeframe with time and money constraints. 

View details →

ISTQB Expert Test Management Strategic Test Manager Training

The Expert Test Management Strategic Test Manager course, created by Rex Black, past President of the International Software Testing Qualifications Board (ISTQB), past President of the American Software Testing Qualifications Board (ASTQB) and co-author of the International Software Testing Qualifications Board Expert Level Syllabus Test Management, is ideal for testers and test teams preparing for certification. 

This hands-on course provides test engineers with the ability to define and carry out the tasks required to put the strategy into action and is ideal for testers and test teams preparing for certification. In preparation for the exam, participants will learn the subject matter behind the test standard and deepen their understanding by working through case studies and exercises. In group exercises, typical review situations are played out and analyzed.

View details →


Copyright ® 2018 Rex Black Consulting Services.
All Rights Reserved.
ISTQB Logo ASTQB Logo IREB Logo PMI Logo ISTQB Logo
PMI is a registered mark of the Project Management Institute, Inc.