Resources

Test-Driven Development, or TDD, is a term used for a popular collection of development techniques in wide use in the Agile community. While testing is part of its name, and though it includes tests, and though it fits in that part of the life cycle usually ascribed to unit testing activity, TDD pundits universally insist that it is not a testing technique, but rather a technique that helps one focus one’s design thinking. The idea is that you write your tests first, and your code second.

Read this article to explore some subtle pitfalls of TDD that have come out of our experience, our consultancy on real projects (all of the conjectured problems are things we have actually seen in practice), and a bit of that rare commodity called common sense.

The original two-part version of this article  about Test Driven Development was published in Better Software Testing.

Continue reading →

A popular article translated into Polish! A number of our clients have adopted Scrum and other Agile methodologies. Every software development lifecycle model, from sequential to spiral to Agile, has  testing implications. Some of these implications ease the testing process. We don't need to worry about these implications here.

Some of these testing implications challenge testing. In this case study, I discuss those challenges so that our client can understand the issues created by the Scrum methodology, and distinguish  those from other types of testing issues that our client faces. 

Continue reading →

Unit testing was a staple of the FORTRAN days, when a function was a function and was sometimes worthy of functional testing. Computers computed, and functions and procedures represented units of computation. In those days the dominant design process composed complex external functionality from smaller chunks, which in turn orchestrated yet smaller chunks, and so on down to the level of well-understood primitives. Each layer supported the layers above it. You actually stood a good chance that you could trace the functionality of the things at the bottom, called functions and procedures, to the requirements that gave rise to them out at the human interface. There was hope that a good designer could understand a given function’s business purpose. And it was possible, at least in well-structured code, to reason about the calling tree. You could mentally simulate code execution in a code review. 

Object orientation slowly took the world by storm, and it turned the design world upside-down. First, the design units changed from things-that-computed to small heterogeneous composites called objects that combine several programming artefacts, including functions and data, together inside one wrapper. The object paradigm used classes to wrap several functions together with the specifications of the data global to those functions. The class became a cookie cutter from which objects were created at run time. In a given computing context, the exact function to be called is determined at run-time and cannot be deduced from the source code as it could in FORTRAN. That made it impossible to reason about run-time behaviour of code by inspection alone. You had to run the program to get the faintest idea of what was going on.

So, testing became in again. And it was unit testing with a vengeance. The object community had discovered the value of early feedback, propelled by the increasing speed of machines and by the rise in the number of personal computers. Design became much more data-focused because objects were shaped more by their data structure than by any properties of their methods. The lack of any explicit calling structure made it difficult to place any single function execution in the context of its execution. What little chance there might have been to do so was taken away by polymorphism. So integration testing was out; unit testing was in. System testing was still somewhere there in the background but seemed either to become someone else’s problem or, more dangerously, was run by the same people who wrote the code as kind of a grown-up version of unit testing.

Classes became the units of analysis and, to some degree, of design. CRC cards (popularly representing Classes, Responsibilities, and Collaborators) were a popular design technique where each class was represented by a person. Object orientation became synonymous with anthropomorphic design. Classes additionally became the units of administration, design focus and programming, and their anthropomorphic nature gave the master of each class a yearning to test it. And because few class methods came with the same contextualization that a FORTRAN function did, programmers had to provide context before exercising a method (remember that we don’t test classes and we don’t even test objects — the unit of functional test is a method). Unit tests provided the drivers to take methods through their paces. Mocks provided the context of the environmental state and of the other methods on which the method under test depended. And test environments came with facilities to poise each object in the right state in preparation for the test.

Continue reading →

I guess it’s time for the second installment. My earlier essay started just as a casual reply to a client, but when Rex Black posted it on his web site, it went viral — going to #3 on Reddit and appearing prominently on other social networking amalgamation sites. Since then I’ve had the pleasure of watching the dialog unfold. It’s ranged from profound to just silly and, sadly, the majority of it falls into the latter category. It appears as though there is a very wide mythology, perhaps propelled by industry hope and fuelled by academic programs and consultants desperate to justify their reason to exist.

There have been precious few real arguments against the positions I laid out in the earlier essay, but a lot of emotive disagreement. In this second round I take inspiration from the dialog that ensued from the first round and offer a few more insights and opinions. Forgive me for opining a bit more in this round than in the first, but I had already covered most of the substantial groundings earlier.

Here, I’ll present a very few fundamental arguments against unit testing that in theory should have been in the first article, such as Weinberg’s Law of Decomposition (An Introduction toGeneral Systems Thinking by Gerald Weinberg, 2001). I’ll offer some models that help us reason about QA in general, such as the Venn diagram on testing opportunity. And I’ve looked further into my contacts to find other smells in unit testing, such as the work-in-progress from Magne Jørgensen, and the broader perspective at Toyota that challenges the very idea of testing as we usually think of it. Last, I’ll look at some of the more archetypical responses that the gallery offered on the first round, together with my analysis. These provide a good cross-section of the typical misunderstandings that flood our industry.

Please refer back to Chapter 1 before launching into social media discussions on Chapter 2. You can find the earlier chapter here.

Continue reading →

Your mission should you choose to accept it: A client needs to implement an integrated, automated unit, component, and integration testing process. The system development teams are spread across 3 continents. They use multiple platforms. Their system architecture is interdependent and complex. Some tests need to be run on an application server. Both static and dynamic testing is required. The process you design must be simple to use, not an additional burden for busy developers. It should be automated and easily integrated. By the way, only 3 designers have been assigned to this mission. You have 6 weeks. Good luck. This message will self-destruct in 20 seconds… 

Continue reading →

As the demand for new technological products grows apace, many companies have adopted Agile methodologies for their software development activity.

The demands on a tester working on projects using Agile methodologies are different to those faced on a traditional software project. Testers on Agile projects, in addition to excellent testing skills, must be able to communicate effectively, interact and collaborate constantly with colleagues, liaise closely with business representatives and work as part of a whole-team responding to early and frequent customer feedback.

To work as part of an Agile software team, testers must first understand the principles and values underpinning the Agile development approach. This section looks at the essential components of Agile software development, explains the advantages of the whole-team approach used in Agile projects and demonstrates the benefits of receiving early and frequent customer feedback.

An excerpt from Agile Testing edited by Rex Black due to be published by BCS Learning & Development in Spring 2017 (subject to change). Section 1.1 by Gerry Coleman. All material is provisional and may be subject to change.

Continue reading →

Continue reading →

Continue reading →

This is an excerpt from my book, Expert Test Manager, written with James Rommens and Leo van der Aalst. I hope it helps you think more clearly about the test strategies you use.

A test policy contains the mission and objectives of testing along with metrics and goals associated with the effectiveness, efficiency, and satisfaction with which we achieve those objectives. In short, the policy defines why we test. While it might also include some high-level description of the fundamental test process, in general the test policy does not talk about how we test.

The document that describes how we test is the test strategy. In the test strategy, the test group explains how the test policy will be implemented. This document should be a general description that spans multiple projects. While the test strategy can describe how testing is done for all projects, organizations might choose to have separate documents for various types of projects. For example, an organization might have a sequential lifecycle test strategy, an Agile test strategy, and a maintenance test strategy.

Continue reading →

[Note: This is an excerpt from Agile Testing Foundations: An ISTQB Foundation Level Agile Tester Guide, by Rex Black, Marie Walsh, Gerry Coleman, Bertrand Cornanguer, Istvan Forgacs, Kari Kakkonen, and Jan Sabak, published July 2017. Kari Kakkonen wrote this selection. The authors are all members of the ISTQB Working Group that wrote the ISTQB Agile Tester Foundation syllabus.]

The traditional way of developing code is to write the code first, and then test it. Some of the major challenges of this approach are that testing is generally conducted late in the process and it is difficult to achieve adequate test coverage. Test-first practices can help solve these challenges. In this environment, tests are designed first, in a collaboration between business stakeholders, testers, and developers.  Their knowledge of what will be tested helps developers write code that fulfils the tests. A test-first approach allows the team to focus on and clarify the expressed needs through a discussion of how to test the resulting code. Developers can use these tests to guide their development.  Developers, testers, and business stakeholders can use these tests to verify the code once it is developed.

A number of test-first practices have been created for Agile projects, as mentioned in section 2.1 of this book. They tend to be called X Driven Development, where X stands for the driving force for the development. In Test Driven Development (TDD), the driving force is testing. In Acceptance Test-Driven Development (ATDD), it is the acceptance tests that will verify the implemented user story. In Behaviour-Driven Development (BDD), it is the behaviour of the software that the user will experience. Common to all these approaches is that the tests are written before the code is developed, i.e., they are test-first approaches. The approaches are usually better known by their acronyms. This subsection describes these test-first approaches and information on how to apply them is contained in section 3.3.

Test-Driven Development was the first of these approaches to appear. It was introduced as one of the practices within Extreme Programming (XP) back in 1990. It has been practiced for two decades and has been adopted by many software developers, in Agile and traditional projects. However, it is also a good example of an Agile practice that is not used in all projects. One limitation with TDD is that if the developer misunderstands what the software is to do, the unit tests will also include the same misunderstandings, giving passing results even though the software is not working properly. There is some controversy over whether TDD delivers the benefits it promises. Some, such as Jim Coplien, even suggest that unit testing is mostly waste.

TDD is mostly for unit testing by developers.  Agile teams soon came up with the question: What if we could have a way to get the benefits of test-first development for acceptance tests and higher level testing in general? And thus Acceptance Test-Driven Development was born.  (There are also other names for similar higher-level test-first methods; for example, Specification by Example (SBE) from Gojko Adzic.) Later, Dan North wanted to emphasize the behaviours from a business perspective, leading him to give his technique the name Behaviour-Driven Development. ATDD and BDD are in practice very similar concepts. 

Let’s look at these three test-first techniques, TDD, ATDD, and BDD, more closely in the following subsections.

Continue reading →

Article from SQ Magazine, www.sq-mag.com
Automated testing continues to be a major transformational factor in software development and there is no doubt that there is an urgent, and rising, requirement for QA and test professionals with automation skills.  In particular, Selenium is globally rated as a top priority in the test automation field.
iSQI has introduced the Selenium Tester Foundation certificate into its portfolio in response to significant market demand for Selenium WebDriver skills. The certification is based on a highly practical hands-on training course that will give an immediate return on investment back in the workplace.
Rex Black, President of RBCS and past President of the ISTQB®, supported the development. The SQ mag asked him about the new course and the certification: 

1. Rex, a new star is born: the A4Q Selenium Tester Foundation. Tell us more!
The A4Q Selenium Tester Foundation is an entry-level training and certification program for people who want to learn and use the very popular test automation tool Selenium. The Selenium Tester Foundation syllabus addresses the key fundamental concepts of application GUI test automation using Selenium WebDriver, which in itself is a really valuable resource for the Selenium community. The three-day training course is exceptionally hands-on, which is critical for test automation, since you can really only learn how to do test automation by doing test automation. The exam for the certification allows people to demonstrate mastery of the concepts in the syllabus, thus providing a way for people to prove their mastery of the fundamentals of Selenium test automation. We think this new star is quite a nice star indeed!
 

Continue reading →

Some people use the terms “verification” and “validation” interchangeably, but there are significant differences between them. Some people disparage verification, or deny that it’s even involved in testing. However, you can’t adequately build confidence and reduce risk in the software you test without using the proper mix of both. In this webinar, Rex will clarify the meaning of these two terms, give examples, and explain why both are essential to proper software testing.

Listen now →

A majority of software development organizations have adopted Agile methods, but testing in Agile projects remains a major challenge. However, some of these organizations have found practical ways to integrating testing into Agile lifecycles.  In this webinar, Rex will address challenges and opportunities associated with Agile, different ways of adapting Agile lifecycles and the testing within those lifecycles, and how Agile differs from traditional lifecycles.  We’ll examine tools and metrics for Agile projects.  We’ll address whether Agile can be used for outsourced projects.  We’ll look at different options for organizing test teams on Agile projects, and why only some of these options can work.  Rex will offer his thoughts on Agile and quality, and what skills and personalities work best in Agile projects. In this updated free webinar, Rex will discuss these points and more, giving you a better shot at Agile testing success.

Listen now →

Agile methods have become widespread over the last 20 years, but it’s taken a while for testing to catch up.  However, Agile testing best practices have emerged, and the ISTQB Agile Working Group has taken on the task of capturing those for you.  In this brief webinar, Rex Black, Chair of the ISTQB Agile Working Group, will explain where the program has been, where it is now, and how it is evolving to support Agile testers in their careers. In twenty minutes or less, you’ll learn how the ISTQB Agile syllabi can help you advance your career as an Agile tester.

Listen now →

For nearly ten years, RBCS has run a highly successful free webinar series.  In 2018, we’re adding the Two Points of View at Two series to our monthly webinar rotation. In each of these sessions, Rex Black will talk with another software luminary about topics of mutual interest, where the two have some different views, and then Rex opens the floor to questions. 

In this inaugural session, Rex is happy to welcome Maaret Pyhäjärvi .  Maaret’s bio describes her as feedback fairy with a day job at F-Secure, where they call her a Lead Quality Engineer. She identifies as empirical technologist, tester and programmer, catalyst for improvement, author and speaker, and community facilitator and conference organizer. You can catch her latest thoughts on her blog at http://visible-quality.blogspot.fi

In this session, Rex and Maaret will discuss tester-developer collaboration and the relationships between testers and developers.  How to approach developers for collaboration? How do testers-developers ratios affect relationships? What about people who move between tester and developer roles? Join Rex and Maaret to hear their thoughts and ask your questions.

Listen now →

The Agile Testing Pyramid is a great metaphor, and it can be very useful when applied properly. However, it can also lead to some serious dysfunctions in organizations that misunderstand, misapply, or misinterpret it.  Are you using the Agile Testing Pyramid properly? Rex will help you answer that question, and help you resolve problems if they exist.  In twenty minutes or less, you’ll learn the rights and wrongs of the Agile Testing Pyramid.

Listen now →

In this month’s “Two Points of View at Two” session, Rex is happy to welcome Nivia S. Henry. Nivia fundamentally believes that happy people, working in a healthy environment, will do great things. This philosophy has driven her to build a 15+ year career creating and supporting high-performing teams. Her career path has included agile coaching, enterprise agile transformations, product management, and people leadership. Today, Nivia applies her hard-earned experience as an Quality and Web Engineering Manager at Spotify. In this session, she'll discuss Spotify's perspective on Quality and focus on how the organization has organically grown its test automation practice.

Listen now →

Selenium has become a very important test automation tool.  Now, the Alliance for Quality (A4Q) has brought out a highly practical, extensively hands-on training and certification, the Selenium Tester Foundation.  In this One Key Idea session, Rex Black, Project Manager for the A4Q Selenium Tester Foundation development project, is here to help you understand this certification and how it can launch you—or boost you along the way—on your career path to becoming a Selenium test automation expert.  In twenty minutes or less, you’ll understand how the A4Q Selenium Test Foundation training and certification can help you progress towards your professional test automation goals.

Listen now →

Archimedes once wrote, “Give me a place to stand, and a lever long enough, and I will move the world.” In Agile, the place to stand is testing, and the lever is test automation. Are you using all the leverage that lever gives you? Most organizations are missing some tricks, and not getting the full value of test automation. In this webinar, Rex will survey all the different ways you can—and should—take advantage of test automation as an Agile tester. The best news: Many of these forms of automation involve tools that won’t cost you a dime! Spend some time learning about some high-ROI, proven best practices to reduce failure costs and increase quality.

Listen now →

We are inundated by scads of mixed messages about software development, testing, quality, and delivery today. There are the Agile Manifesto and principles, SCRUM, Kanban, Scaled Agile Framework, the book - How Google Tests Software, the tech talk - Test is Dead, murmurs about achieving 100% automation, goals of delivering faster and more frequently to customers, the idea that we can solve every problem with DevOps, and so on. Some of these messages leave organizations confused about the value of testing, and testers wondering if they have a career path at all. We indeed are at a critical juncture in the quality and testing space, and if we aren't careful, we could be joining the dodo and the dinosaurs. 

How will we survive all this? Dawn believes the heart of the issue is in becoming truly agile in our beliefs, approaches, and attitudes about testing. Without the flexibility to serve the needs of our teams and organizations TODAY (and tomorrow!), we should be concerned. So, what is the true meaning of agility for testers? Does it mean doing "Agile" things? Following SCRUM? Doing DevOps? Etc.? Dawn doesn't think so. Please join Dawn to take a deep dive into what true agility could look like for testers and teams moving forward!

Listen now →

Perhaps you, like many software testers and test managers, are enjoying the benefits of incorporating a lightweight analytical risk-based testing strategy such as RBCS’ Pragmatic Risk Analysis and Management (PRAM) technique into your test approach.  What if your organization is adopting an agile methodology?  Can you still use PRAM and other similar analytical risk-based testing strategies?  Absolutely.  In this webinar, Rex will explain the process for integrating quality risk identification and assessment into the release and iteration planning processes, and for integrating quality risk mitigation into each iteration.  You’ll return to work with a proven method for injecting risk-based testing into your agile processes.

Listen now →

Listen now →

Is the V model dead, crushed by the onset of agile methods?  In the age of agile, are those who talk about the V model anachronistic dinosaurs?  Or was the V model actually ahead of its time, illuminating best practices that now form the core of agile testing?  In this talk, Rex will show how the fundamental principles of the V model can be successfully applied in the context of an agile development effort, and how the application of those principles make agile software development even more agile.  Sure to stir some controversy, attend Rex’s webinar, submit questions and comments, and make up your own mind.

Listen now →