Getting better at Domain Testing

In late January of 2014, after the Workshop on Teaching Software Testing (WTST) at Florida Institute of Technology, Dr. Cem Kaner and Dr. Rebecca Fiedler put together a 5-day pilot course to beta test a new Black Box Software Testing (BBST) course called Domain Testing. I was one of ten participants to try it out.

Then they announced a follow up / pilot class for BBST Domain Testing that would be online. I applied to be part of the class and was accepted. This was like an expanded version of the pilot course (less intense than 5 full days) but more similar to what you might expect from a BBST courses. The course focused from Day One on diving deeper into the test technique Domain Testing and working through various aspects of the Domain Testing schema until we got to a final / capstone project where we put everything together.

I’m happy to share that I’ve officially completed the course!

I’m an AST BBST Instructor

That’s right, I completed the trifecta of BBST classes and decided to continue on to being an Instructor. I finished the class in November but just recently got the proof:

Boom! Now comes the hard work – working as an assistant through enough classes until I learn the system and work my way up to a lead instructor role for AST.

Part of my desire for taking the BBST Instructor class is to become more familiar with material taught in the BBST classes, get exposed to new student’s ideas (that will hopefully broaden and challenge my understanding) and learn how to provide important and useful feedback (and help others do the same thing).

Eventually I hope to teach the BBST materials to people I work with and help expose others to the material outside of AST. This was the first step.

I’m a Bug Advocate

I do advocate for bugs to be fixed but the title comes from passing the Association for Software Testing‘s (AST) Black Box Software Testing (BBST) Bug Advocacy course. The class officially ended in mid-July and it marks the third and final BBST class for me. Together Foundations, Bug Advocacy and Survey of Test Design make up a semester long class that Cem Kaner teaches at Florida Institute of Technology called Software Testing 1. These lectures and classes are, as far as I know, the only college level software testing courses available to anyone that wants to learn about testing – the material is free and the courses are pretty inexpensive.

As with the prior classes I agreed to have my name listed on AST’s graduates website and for fun my certificate of completion is below:

Despite this being the final BBST class my journey with BBST doesn’t end here. I enrolled in the Instructor class for November at the beginning of the year when I enrolled in Test Design. I figured the best way to become more familiar with the material and reinforce what I do know (challenge what I don’t know) is to try to teach it. Worked for Scuba Diving!

I’m a Black Box Test Designer

More accurately I’ve passed the Association for Software Testing‘s (AST) Black Box Software Testing  (BBST) Test Design class. Test Design is a survey class, students are introduced to 30+ types of tests but there is only enough time to focus on a few of them: risk-based testing, specification-based testing and domain testing. In December I passed the Foundations course and a few days ago I started the Bug Advocacy course (there are 3 courses in the BBST series) which sets me on the path to meet my goal of taking all 3 courses this year. I also signed up for the Instructors course towards the end of the month, wahoo!

I agreed to have my name listed on AST’s graduates website but just for fun here is my certificate of completion:

The Test Design instructors thought I understood enough of the information to pass me. Test Design was like the Foundations class – a lot of reading, labs and collaboration. In the time between Test Design ending and Bug Advocacy starting I cleaned up some of the labs but I haven’t finished. My goal will be to turn these labs (work products) to build out my testing portfolio. Some day I might even make that portfolio public so others can see what I’ve done and perhaps use it as inspiration to build their own.

NRG Global Test Competition Retrospective

Roughly two and a half weeks ago I competed in the first NRG Global Test Competition. The idea behind the competition was simple: get a bunch of people/ teams together to test a few products, split the competition into two days, one with functional testing and another with performance testing, and based on the reports submitted judges would award points and announce winners. The full details are available here and here.This was the first online testing competition I’d tried but thanks to my experiences with testing challenges and rapid testing online I knew I’d have fun once I got past the quirks. By quirks I mean it can take time to get comfortable with the discussion format, figure out how to ask questions, how best to communicate with my fellow team members, etc. The competition took place at 10 am Eastern which sucks if you live on the west coast and have to wake up before 7 am like I did. It was all for fun anyways.

For as early in the morning and as new as the competition was I think I did reasonable. Not great, not even good, but reasonable. I think the best way to phrase it is: I’m not happy with my work. (I might be overly critical here but still.) Now we only had 3 hours from introduction of the products under test, to learn the product, ask questions of the “owner”, test it, ask more questions, file bugs and write a report. Yet when I think back at what we turned in I’m not happy with it. Let me explain.

My team member and I barely communicated with one another. We were using Skype but we didn’t do much planning ahead of time (not like we could have because nothing was public) so when it came time for the competition it was a simple “hi”, “what are you working on” and “I’ll look at x”. That was it. We each went to different applications. Thinking back on it now I think we would have done much better if we were on the same application, talking to one other about what we were seeing. My experience has been any collaboration no matter how small results in finding and learning amazing things.

At the time of competition I considered using Bach’s HTSM to map out the application but didn’t. I wish I had. Even though its a bit detailed and were we on a short deadline I think heuristics would have lead me to think about and discover even more potential problems. At the very least I’d feel more confident in what I had tested.

It took me an hour or so to really get started looking at the products, deciding which one to test and with what equipment (iPad), browsers, etc. I’m still blaming the time of day. I started with some simple touring of this “home built” application that must have been built specifically for the competition because it was really simple and full of problems. Even though I saw lots of problems initially I took note and kept searching until I felt I had covered the entire application as best as I could. Then I circled back, asked a few questions from the “product owner” Matt Heusser and began testing the problems I saw. By that time I had just enough time to get my bugs written up in the tracking system  (I had maybe 5) and started working on our Test Report. I think we got our report in right at the deadline.

I wasn’t able to commit to the second part of the competition, for performance testing and I’m not sure if my team member was able to either. I knew going in I couldn’t commit time for it however I’m still holding on to hope that I’ll get a chance to play with the AppLoader tool. Despite my displeasure with my performance I’m glad I joined, in fact I’m looking forward to getting the feedback on how other’s think we did. =)

I’m a Black Box Software Tester

More accurately I should say I’ve passed the Association for Software Testing‘s Black Box Software Testing Foundations class.Here’s the proof:

What does this mean? It means the instructors think I understood enough of the material to pass me based on the work I did throughout the course which includes discussion forums, assignments, quizzes and a final examination. I feel like I learned a lot from the exercises, readings and watching Cem Kaner’s videos.

For those who don’t know, Foundations is an intense 4 week class covering the basics of black box testing including the mission of testing, the oracle problem, the measurement problem and the impossibility of complete testing. I’d definitely recommend the class as long as you can spare at least 12+ hours a week to commit to watching the videos, reading the required and recommended readings and participating in the assignments. You must take the Foundations course before you can take any further classes so you learn how they are run.

One of my goals for 2013 is to take the other courses: Bug Advocacy, Test Design and then Instructor!

My Tester’s Commitments

My job is to help programmers look good; to support them as they create quality; to ease that burden instead of adding to it. In that spirit, I make the following commitments:

  1. I provide a service. You are an important client of that service. I am not satisfied unless you are satisfied.
  2. I am not the gatekeeper of quality. I don’t “own” quality. Shipping a good product is a goal shared by all of us.
  3. I will test your code as soon as I can after you deliver it to me. I know that you need my test results quickly (especially for fixes and new features).
  4. I will strive to test in a way that allows you to be fully productive
  5. I’ll make every reasonable effort to test, even if I have only partial information about the product.
  6. I will learn the product quickly, and make use of that knowledge to test more cleverly.
  7. I will test important things first, and try to find important problems. (I will also report things you might consider unimportant, just in case they turn out to be important after all, but I will strive to spend less time on those.)
  8. I will strive to test in the interests of everyone whose opinions matter, including you, so that you can make better decisions about the product.
  9. I will write clear, concise, thoughtful, and respectful problem reports. (I may make suggestions about design, but I will never presume to be the designer.)
  10. I will let you know how I’m testing, and invite your comments. And I will confer with you about little things you can do to make the product much easier to test.
  11. I invite your special requests, such as if you need me to spot check something for you, help you document something, or run a special kind of test.
  12. I will not carelessly waste your time.

Inspired by A Tester’s Commitments.