Blog

Using Metrics Can Be Smart, As Long as You're Not Stupid with Them

f you have been in software engineering for a while—or in fact just in the working world in general for a while—you’ve probably seen someone do something stupid with metrics. Such mistakes raise a whole bunch of interesting questions. What are the most common metrics mistakes? Why are they mistakes? Why do people make these mistakes? Are you making these mistakes? Why use metrics at all, when there are so many mistakes? In this recorded webinar, I'll give real-world examples of these mistakes, explain the management and economic theories behind metrics, and help you find ways to implement metrics that aren’t stupid.

Need more help with metrics? Contact us. We just worked with a client and identified around $15,000,000 in efficiency improvements using their incident metrics, a 1000x return on the cost of having us come in and do the work.

— Published


More Help for Agile Testers and Other Agile Professionals

Just in time for the US Independence Day holiday, my thirteenth book, Agile Testing Foundations, is available now. Agile testers, developers, scrum masters, test managers, product owners, and candidates for the ISTQB Agile Tester Foundation exam, this book is for you. I thank my co-authors for their hard work on this book with me.

Wondering what's next?  Well, book number fourteen, Mobile Testing Foundations, will be out later this year. It covers the ASTQB Certified Mobile Tester exam, and is currently in the review process. 

— Published


Calling All Mobile Testers

If you are testing mobile devices and mobile apps--and, at this point, who isn't?--you'll have another useful resource by the end of this year.  You're welcome.

— Published


Don't Let Your Testing Go Pear-shaped; Get Wise about Pairwise Testing: Part 3

When you are doing certain types of testing, there are a relatively small number of factors, each with a relatively small number of options, that are not supposed to interact. For example, consider testing a browser-based application on a PC or Mac. You may have five or six browsers, four or five OS versions, six or seven anti-malware packages, a few browser plug-ins, and three or four connection speed options. If you do the math, though, there are thousands of combinations. You can't test everything, but you want to go beyond basic equivalence partitioning. What to do?  

In this classic RBCS webinar from our One Key Idea series, I explain how to use a tool from NIST to generate pairwise testing tables. It goes beyond simpler tools, because it allows for constraints, mixed levels of combinations, and more.  Better yet, this tool, ACTS, is available worldwide for free. 

Need more help designing effective and efficient tests? Contact us. At RBCS, we've been helping people test better for over two decades, and we can help you.

— Published


Twelve Ways to Be an SDET that Don't Involve Test Automation

Of course, a big part of being a Software Development Engineer in Test (SDET)/Software Engineer in Test (SET) is test automation, but there's plenty more involved. Consider the following 12 activities:

  1. Teach a developer what statement and decision coverage are
  2. Teach a developer what MC/DC coverage is and why it’ll find bugs statement and decision coverage won’t
  3. Use dynamic analysis tools during your tests to watch how they perturb the system
  4. Use a sniffer to watch for sensitive data transmitted in the clear across a network
  5. Use SQL queries to make sure that what the GUI told you happened actually happened at the data layer
  6. Run a leak detection test on that stupid app that keeps crashing every few days
  7. Use a fault injection/fuzzing tool to corrupt configuration and/or data files before running your tests
  8. Analyze the interfaces in your app/systems to do a (partial) integration test coverage analysis
  9. Analyze the data sources and sinks used by your app/system to do another (partial) integration test coverage analysis
  10. Use complexity analysis and defect data to produce a hot-spot analysis of your code, especially for regression bugs
  11. Evaluate what lessons you can learn from the Challenger disaster to be a more effective SDET/SET/technical tester
  12. Laugh out loud in utter disbelief when you hear someone say “testing is dead”

Think that's a lot? Actually, I could easily add another dozen to this list. The main morale of this post is: If you're an SDET, you are a test automation engineer, yes, but also a whole lot more.

— Published


An Open Letter to Readers of Technical Books

I received a nice message from a LinkedIn connection, thanking me for my books.  Here's my response: 

"Thanks. The greatest compliment you can give authors of technical books such as me is to actually buy the books and not ever use, distribute, or encourage the use of pirated copies of the books that are on the internet. Authors of technical books like me make almost nothing from those books, but what little money we do make does help. Seeing people steal those books and post them all over the internet is a real insult to the efforts we put into writing them. Thanks again for your kind words and support.
Regards,
Rex"

Just in the last year, I have had to deal with four blatant examples of intellectual property theft, and I didn't even have to go looking for the stolen materials. If I tried right now, I could find pirated copies of my four best-selling books on the internet with one search each.

I would ask everyone who appreciates technical books and finds them useful to keep this in mind. Stealing the meager royalties associated with such books from the people who work long hours creating those books is hardly a way to repay the favor. 

Thanks,
Rex

— Published


Don't Let Your Testing Go Pear-shaped; Get Wise about Pairwise Testing: Part 2

When you are doing certain types of testing, there are a relatively small number of factors, each with a relatively small number of options, that are not supposed to interact. For example, consider testing a browser-based application on a PC or Mac. You may have five or six browsers, four or five OS versions, six or seven anti-malware packages, a few browser plug-ins, and three or four connection speed options. If you do the math, though, there are thousands of combinations. You can't test everything, but you want to go beyond basic equivalence partitioning. What to do? 

The standard solution is pairwise testing, but can pairwise testing be misused? In this classic RBCS webinar, I explain the most pervasive myths associated with pairwise testing, which can lead to over-use, under-use, and misuse of this powerful technique. Give a listen and get wise about pairwise! 

Need more help designing effective and efficient tests? Contact us. At RBCS, we've been helping people test better for over two decades, and we can help you.

— Published


Don't Let Your Testing Go Pear-shaped; Get Wise about Pairwise Testing: Part 1

During my recent presentation of our Advanced Black-box Bug-a-thon here in Milan, people were very interested in pairwise testing. What is it? What kinds of problems can it solve? How do I avoid the pitfalls of using it? What tools are available? In this three-part series of classic RBCS webinars, I'll address all these questions and more.

First, let's define the problem we're trying to solve. When you are doing certain types of testing, there are a relatively small number of factors, each with a relatively small number of options, that are not supposed to interact. For example, consider testing a browser-based application on a PC or Mac. You may have five or six browsers, four or five OS versions, six or seven anti-malware packages, a few browser plug-ins, and three or four connection speed options. If you do the math, though, there are thousands of possible combinations. You can't test everything, but you want to go beyond basic equivalence partitioning. What to do? 

In this first classic RBCS webinar, we'll examine what pairwise testing is all about and how it can help solve these kinds of problems. Need more help designing effective and efficient tests? Contact us. At RBCS, we've been helping people test better for over two decades, and we can help you.

— Published


First-ever ISTQB Advanced Security Tester Training Yields Perfect Result

Earlier this month, we ran the first-ever ISTQB Advanced Security Tester course with our partners imbus at their offices in Erlangen, Germany (near Nuremberg). It was a great experience, with excellent feedback from the attendees.

Now, the exam results are in: 100% pass rate. That's rate, all eleven attendees passed the exam, based on their hard work and the course materials. Regarding the training delivery, I'll let one of the attendees give his appraisal of that, "No problem with such a good trainer."

With security risks to applications and systems growing day by day, you really need to be aware of the best practices of software and system security, regardless of your role, whether a security tester, a functional tester, an SDET/SET, a test manager, a product owner, or a developer. With our three-day Advanced Security Tester course, we can teach you those best practices, and, based on these exam results, give you an excellent chance of passing the exam.

Take a listen to my short message on the risks you face, and then contact us today for more details on how to schedule this course at your location.

— Published


Immediate Results from Expert Test Manager Training

I received some immediate feedback from Kapros Gabor who attended last week's ISTQB Expert Test Manager course, "Your training was really thought-provoking for me. Next days I am creating a real automation strategy for my system testers team – I took a liking for creating that. We will have a management review on 06.July, so I have a rather strict deadline to create and present it. :-)"  

It's always good to hear of such a quick, measurable benefit from our courses. All of our training includes extensive hands-on components to enable quick transfer of lessons learned from the training to the real world. If you're interested in an Expert Test Manager course, or any of our courses, contact us.

— Published



Copyright ® 2019 Rex Black Consulting Services.
All Rights Reserved.
ISTQB Logo ASTQB Logo IREB Logo PMI Logo ISTQB Logo
PMI is a registered mark of the Project Management Institute, Inc.