Test Automation Case Study
RBCS helped a medical device client, Arrowhead Electronic Healthcare, apply a creative approach to test automation called a dumb monkey. Arrowhead’s primary product is a patient diary, an application that runs on a handheld device and captures patient information during drug trials. As such, it has complex menus and dialog boxes, presenting to the tester an almost-infinite input space. A dumb monkey is a tool that allows automated exploration of this large input space in search of various types of bugs. The idea of swinging randomly through the input space is where the monkey part of the name comes from. The word dumb arises from the fact that the tool can only spot certain types of bugs. However, dumb monkeys can be built quickly and easily, often from freely-available materials.
To help Arrowhead realize quick, cheap benefits from test automation, we worked with them to design a dumb monkey to address the following quality risks:
- Reliability
- Translation completeness
- Functionality of the user interface
- Input error checking
- Verification of requirements
The dumb monkey was built entirely from freeware parts downloaded from the Internet, mostly perl and bash scripts running on Linux. The tool budget was zero, and yet only 120 hours of programmer effort was required to build the monkey. The architecture is clean and maintainable; indeed, the monkey is currently scheduled to migrate into the Internet cloud to provide extended reliability and load testing.
The Arrowhead team, from the executives to the individual testers and developers, love their dumb monkey. They report that “[while] using the Monkey over a four month period, we noticed significant time [and effort] savings, mainly in the areas of diary testing, screenshot capturing, and translation verification. The Monkey allows the compression of two calendar days of functional testing into a single half-day. This allows for flexibility and changes during the test period. The [effort] saved doing translation verification [alone] for a single diary...was approximately 323 hours, obviously surpassing the 120 [development] hours. Since the Monkey is diary independent, our return on investment will continue to grow.” Click here to read more about this case study.
All Rights Reserved.
All Rights Reserved.
PMI is a registered mark of the Project Management Institute, Inc.