Review: Exploratory Software Testing – Tips, tricks, tours and techniques to guide test design

Some of the first testing books I read were from James Whittaker’s How to Break Software series. Those books, like this one, are laid out in a practical manner with each chapter focused on a specific attack or approach making them easy to read, reference and apply. Perfect for learning. I picked up this book a few years ago when I started questioning the way I was testing. The material was new to me and made me ask what is exploratory testing and what does touring have to do with it?

According to Whittaker (pg. 16) exploratory testing (E.T.) is testing where scripts or rigidity have been removed (paraphrasing). Whittaker explains his terms “E.T. in the small”, decisions made where the scope of the testing is small and “E.T. in the large”, decisions made when the scope of testing is large (small might be a screen in an application while large is the whole application). At the end of chapter 3 he mentions E.T. can be done in a way that allows test planning and execution to be completed simultaneously which is one of E.T.s most important aspects and simplest definitions. Touring (as in a tour guide or sight-seeing) becomes a metaphor for and a way to structure E.T.

There are eight chapters in the book plus a number of appendices. In the first few chapters Whittaker discusses what he sees as the case for software quality (the context of the book), introduces E.T. and explains how he uses it, in the small and the large. The next four chapters cover tours he and others have come up with. The last chapter is about how Whittaker sees the future of testing or at least how he did at the time of publishing.

The first appendix, A, is one of the most important parts of the book: building a successful career in software testing. Whittaker talks about how he got into testing and gives some advice on “getting over the hump” to be a better tester. Appendix A is short but worth reading. The rest of the appendices are old blog posts from his Microsoft days.

As a beginner I found this book much more valuable than I do now several years later. I understand E.T. is an approach to testing that can but doesn’t necessarily include tours or scripts. It isn’t just manual testing either. For reference Michael Bolton (the testing expert) has some good posts in what E.T. is not: (notice how the first post is about touring?)

As you might not guess from the title of this book it does not do a proper job explaining E.T. in a way that one can use it, aside from following the tour metaphor. In fact after reading it again this book seems to say to the reader: these tours are the best, don’t you agree? It’s important to understand exploratory testing is about the way you work, and the extent to which test design, test execution, and learning support and reinforce each other.

According to James Bach the term “exploratory testing” was coined and first published by Cem Kaner and has been worked on by Bach, Whittaker and Kaner (among others) over the last decade. It seems a bit odd that in a book about E.T. Whittaker never mentions their work and provides no references for the reader to follow up. Apparently Whittaker thinks the easiest way to explain E.T. is through testing tours (hence the book) while Bach has a more direct explanation of what constitutes exploratory testing. I found Bach’s post more informative, applicable and frankly cheaper than Whittaker’s Exploratory Software Testing book.

Exploratory Software Testing (the book) offers a limited metaphor for understanding exploratory testing. It isn’t as practical as Whittaker’s previous books because you can’t apply the teachings very well without fully understanding what E.T. is and how tours fit in. If you only want ideas on how Microsoft’s testers used the touring metaphor to “perform” exploratory testing then you’ll get four chapters of information otherwise Exploratory Software Testing is worth skipping.

I wrote the same review on Amazon under the heading “Limited metaphor for exploratory testing”.

My Tester’s Commitments

My job is to help programmers look good; to support them as they create quality; to ease that burden instead of adding to it. In that spirit, I make the following commitments:

  1. I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied.
  2. I am not the gatekeeper of quality. I don’t “own” quality. Shipping a good product is a goal shared by all of us.
  3. I will test your code as soon as I can after you deliver it to me. I know that you need my test results quickly (especially for fixes and new features).
  4. I will strive to test in a way that allows you to be fully productive
  5. I’ll make every reasonable effort to test, even if I have only partial information about the product.
  6. I will learn the product quickly, and make use of that knowledge to test more cleverly.
  7. I will test important things first, and try to find important problems. (I will also report things you might consider unimportant, just in case they turn out to be important after all, but I will strive to spend less time on those.)
  8. I will strive to test in the interests of everyone whose opinions matter, including you, so that you can make better decisions about the product.
  9. I will write clear, concise, thoughtful, and respectful problem reports. (I may make suggestions about design, but I will never presume to be the designer.)
  10. I will let you know how I’m testing, and invite your comments. And I will confer with you about little things you can do to make the product much easier to test.
  11. I invite your special requests, such as if you need me to spot check something for you, help you document something, or run a special kind of test.
  12. I will not carelessly waste your time.

Inspired by A Tester’s Commitments.