Better Tester Training Materials

Photo by Clark Tibbs on Unsplash

Last month the Association for Software Testing (AST) announced a new partnership with Altom, the owner of BBST®, that enables the AST to refresh our curriculum lineup with the new BBST® Community Track and help fund the future growth of the materials. This partnership and refresh are a huge milestone for the AST. 

For some perspective: Cem Kaner developed the BBST courses over a long period of time, starting with Hung Nguyen in the 90s. In the 2000s after Cem Kaner was recruited to Florida Institute of Technology he and Becky Fielder received grants from the National Science Foundation to adapt them into online courses. Some time later Cem began collaborating with the AST to teach and develop the courses further. Those courses became known as AST-BBST to differentiate the way they developed and were taught (by passionate volunteers). Eventually Cem formed his own company, developed BBST® further and sold it to Altom after he retired. 

BBST® classes are well known for their depth of core testing knowledge and focus on improving through peer review work. That’s great for students but frankly it’s a maintenance challenge. Long before I started with the AST the problem has been, how do we properly maintain and evolve the materials and classes?

(more…)

Getting better at Domain Testing

In late January of 2014, after the Workshop on Teaching Software Testing (WTST) at Florida Institute of Technology, Dr. Cem Kaner and Dr. Rebecca Fiedler put together a 5-day pilot course to beta test a new Black Box Software Testing (BBST) course called Domain Testing. I was one of ten participants to try it out.

Then they announced a follow up / pilot class for BBST Domain Testing that would be online. I applied to be part of the class and was accepted. This was like an expanded version of the pilot course (less intense than 5 full days) but more similar to what you might expect from a BBST courses. The course focused from Day One on diving deeper into the test technique Domain Testing and working through various aspects of the Domain Testing schema until we got to a final / capstone project where we put everything together.

I’m happy to share that I’ve officially completed the course!

Conference Idea – A Bug Advocacy Boot Camp?

If someone asked you to speak at or submit a proposal to speak at a conference do you know what you’d say?

I’d say no thanks for now, but I do have an idea for a tutorial. (This is a not-so-well thought out brainstorm and not a proposal.)

Anyone who’s read my blog of late knows I’m a fan of the Black Box Software Testing courses and anyone who has taken a course knows there’s more material cover than the typical 4 week class can get to. It’s very good material but it’s overwhelming at times. One of the classes, Bug Advocacy, has an immediate real world impact for practitioners. If the class could be sliced up in a way that helped participants understand:

  • Basic concepts (quality, bug workflow) 
  • The value of writing good bug reports for the bug writers (testers, programmers, etc.) 
    • For the Bug Reporters – testers primary work product and they put our credibility on the line 
    • For the organization – Well written bug reports can have an impact on quality and the development lifecycle 
    • Motivating the Bug Fixer
  • How to Write a Good Bug Report 
  • This boot camp is not a supplement to the full Bug Advocacy course

I think it could be a valuable Full Day Tutorial. Not as valuable as taking the full class but still valuable for a professional conference.

The challenges are numerous. This is typically a four week course with six recorded lessons, hundreds of slides, a final written exam and a few assignments that include requiring students to analyze a product and write persuasive, well written bug reports. The class also requires students to join the Open Office project, select a few poorly written bug reports, analyze them and submit re-writes that are better. There’s also a lot of time for peer-review and comparisons of well written and not so well written bugs.

Perhaps by focusing more on the Learning Objectives and less on content with some staged / preplanned simple and complex assignments would work?

This idea is interesting for a number of reasons:

  • It would further my understanding of the material and how to teach it (instructional design) which is part of the reason I became a BBST Instructor (I’d like to one day teach the material on my own). 
  • Bug Advocacy has enough practical material that it would be compelling to students who may not be aware of the BBST classes.
    • Further raise awareness for BBST classes and those who teach it (AST & KFA) 
  • Get more people involved in the open courseware community and potentially lead to better material.

I’d want a number of trial runs with this new instructional design before I applied to a conference. Perhaps after a few classes in my new role as an AST-certified BBST Instructor and then by some willing participants at a full day meet up I might have a better idea of the viability of a boot camped / introduction to Bug Advocacy. Maybe there would be a way to include the eventual Bug Advocacy Workbook in the class or as a way for the students to follow up. (Note the Foundations Workbook has just been released and I have no idea when the Bug Advocacy one will be out.)

Any ideas or advice?

(Thanks to Becky Fiedler who suggested Bug Advocacy as the BBST class most appropriate for a full day conference during a discussion we had during WTST / BBST Domain Testing last month.)

I’m an AST BBST Instructor

That’s right, I completed the trifecta of BBST classes and decided to continue on to being an Instructor. I finished the class in November but just recently got the proof:

Boom! Now comes the hard work – working as an assistant through enough classes until I learn the system and work my way up to a lead instructor role for AST.

Part of my desire for taking the BBST Instructor class is to become more familiar with material taught in the BBST classes, get exposed to new student’s ideas (that will hopefully broaden and challenge my understanding) and learn how to provide important and useful feedback (and help others do the same thing).

Eventually I hope to teach the BBST materials to people I work with and help expose others to the material outside of AST. This was the first step.

The Domain Testing Workbook is available

Cem Kaner, Sowmya Padmanabhan and Doug Hoffman have a new book called The Domain Testing Workbook. I’d highly recommend picking up a copy or at least adding it to your reading list! This book is not just a deep dive into one test technique but it represents a collective thinking about what software testing is today.

Domain Testing

BBST Test Design was my formal introduction to Domain Testing aka boundary and equivalence class analysis. Domain Testing is often cited as the most popular (or one of the most popular) test techniques in use today. For its part the Test Design course spends a whole week, a full lecture series and at least one assignment introducing and practicing this technique. If you’d like an introduction I recommend the first lecture from the fifth week in the Test Design series in which Cem introduces Domain Testing:

(For more information see the Testing Education Website or YouTube.)

I got the chance to do some reviewing of the workbook and enjoyed how it elaborated on the material I was learning Test Design. Yet it went further by offering layers of detailed examples to work through and is set up to allow the reader to skip around to whatever section or chapter they find interesting.

In an email, Cem described the book:

My impression is that no author has explored their full process for using domain testing. They stop after describing what they see as the most important points. So everyone describes a different part of the elephant and we see a collection of incomplete and often contradictory analyses. Sowmya, Doug and I are trying to describe the whole elephant. (We aren’t describing The One True Elephant of Domain Testing, just the one WE ride when we hunt domain bugs.) That makes it possible for us to solve many types of problems using one coherent mental structure.

Cem Kaner

I’m excited for this book to be turned into a class on domain testing ( and hopefully open sourced) with the rest of BBST. In the meantime pick up the book from Amazon and let me know what you think!

I’m a Bug Advocate

I do advocate for bugs to be fixed but the title comes from passing the Association for Software Testing‘s (AST) Black Box Software Testing (BBST) Bug Advocacy course. The class officially ended in mid-July and it marks the third and final BBST class for me. Together Foundations, Bug Advocacy and Survey of Test Design make up a semester long class that Cem Kaner teaches at Florida Institute of Technology called Software Testing 1. These lectures and classes are, as far as I know, the only college level software testing courses available to anyone that wants to learn about testing – the material is free and the courses are pretty inexpensive.

As with the prior classes I agreed to have my name listed on AST’s graduates website and for fun my certificate of completion is below:

Despite this being the final BBST class my journey with BBST doesn’t end here. I enrolled in the Instructor class for November at the beginning of the year when I enrolled in Test Design. I figured the best way to become more familiar with the material and reinforce what I do know (challenge what I don’t know) is to try to teach it. Worked for Scuba Diving!

Why take a Black Box Software Testing course?

I was recently telling a friend about the BBST Bug Advocacy course I was working on and he asked why I was taking a black box testing course. I think what he meant was why would I take a course on black box testing as opposed to glass box (or white-box) testing?

This is the more thoughtful answer I wish I had given.

I fell into the testing profession. The University I went to taught some technical skills as part of the degree (programming, database design, etc.) but I never learned anything I would consider fundamental to understanding software testing, nothing that helped me deal with testing problems. Since I work in software testing I wanted to learn more about the domain, to better understand it and be able to differentiate between testing and other types of problems (technology, communication, requirements, etc.). It just so happens the courses that focus on the domain of software testing also focus on black box system level testing.

Black box and glass box testing approaches focus on different things. Black box is testing and test design without knowledge of the code. A black box tester will approach testing based on the interactions of the stakeholders with the system’s inputs and outputs. Contrast this to glass box as testing and test design with knowledge of the internal code. The glass box tester will approach testing based on the interactions of the underlying code, as in “does this code do what the programmer intended it to do?”

Testers using either approach can benefit from understanding the domain of software testing. Understanding what oracles are, how they are heuristics, advantages / disadvantages to measurement, how to communicate and write bugs effectively or even how to design appropriate tests affect both the black box and glass box approaches. That’s why someone would take a black box testing course (specifically the BBST series of courses); that’s why I did.

I’m a Black Box Test Designer

More accurately I’ve passed the Association for Software Testing‘s (AST) Black Box Software Testing  (BBST) Test Design class. Test Design is a survey class, students are introduced to 30+ types of tests but there is only enough time to focus on a few of them: risk-based testing, specification-based testing and domain testing. In December I passed the Foundations course and a few days ago I started the Bug Advocacy course (there are 3 courses in the BBST series) which sets me on the path to meet my goal of taking all 3 courses this year. I also signed up for the Instructors course towards the end of the month, wahoo!

I agreed to have my name listed on AST’s graduates website but just for fun here is my certificate of completion:

The Test Design instructors thought I understood enough of the information to pass me. Test Design was like the Foundations class – a lot of reading, labs and collaboration. In the time between Test Design ending and Bug Advocacy starting I cleaned up some of the labs but I haven’t finished. My goal will be to turn these labs (work products) to build out my testing portfolio. Some day I might even make that portfolio public so others can see what I’ve done and perhaps use it as inspiration to build their own.

What I’ve been up to lately

Things have been busy in the last month or so and I felt like sharing what I’ve been up to lately. Most of it revolves around software testing:

April saw the start of Dan Ariely’s A Beginners Guide to Irrational Behavior class on Coursera. I knew I had the BBST course coming up so I didn’t commit much time to the class other than watching the video lectures and doing the video quizzes. There are many aspects of irrational behavior that affect what we do in software development and testing – I’d like to write a more in-depth article about that in the future.

On the 14th of April I started the BBST Test Design course and completed it on May 8th. For those who have never taken a BBST class before they are incredibly intense month long courses. The course breaks a single calendar week into 2 class weeks – one week with 4 days, and a shorter week with 3 days and each week requires about 10-15 hours of work in order to do the readings, labs and work on the exam. The class is done but I still don’t know if I’ve passed; regardless I learned a lot.

On April 19th I joined the NRG Global Online test competition. My last post was a reflection on how well I thought I did and despite my low perception, my team ended up winning part of the competition!

I went to STPcon 2013 at the end of April in San Diego where I met up with a few Miagi-Do’ers, met some other testers I’d heard from in the twitter-verse or blog-o-sphere and learned a few things. I’m planning to write an experience report and post it either here or on the newly formed Miagi-Do blog. I think it might apply a little more here but I don’t know how it will turn out because I haven’t written it.

During the Test Design course I picked up on Test Design being the last of the 3 BBST courses and there being 3 more courses – Domain testing, spec-based testing and scenario-based testing listed in Cem Kaner’s diagram. I asked Cem about the domain testing course over twitter and he kindly sent me an email with a draft domain testing workbook which I plan to review – right after I email him back and telling him when.

May 8th through 10th I participated in the Rapid Testing Intensive Online #2 as a peer reviewer. It was fun to sit on the other side and provide some feedback to the students on their work although I would have been more effective if I was able to do the assignment as the students were – I just couldn’t take the time off work. Nevertheless I found participating as a peer reviewer to have its own unique challenges as I interacted with other testers and tried to answer their questions. In the RTIO there’s a ton of material and references coming at the students so it helps to interact and help others.

May 16th I signed up for the BBST Bug Advocacy class that takes place in June. One of my year end goals is to complete all 3 BBST courses and then pursue BBST Instructor so I can help others. In fact as I was writing this I signed up for the BBST Instructors course in October!

Lastly I’m looking for a cheap / free place to host a public Rapid Software Testing course with Paul Holland in the Los Angeles area. Anyone know of a place that can fit 20 people comfortably?

Wow I’ve got a lot to do…

I’m a Black Box Software Tester

More accurately I should say I’ve passed the Association for Software Testing‘s Black Box Software Testing Foundations class.Here’s the proof:

What does this mean? It means the instructors think I understood enough of the material to pass me based on the work I did throughout the course which includes discussion forums, assignments, quizzes and a final examination. I feel like I learned a lot from the exercises, readings and watching Cem Kaner’s videos.

For those who don’t know, Foundations is an intense 4 week class covering the basics of black box testing including the mission of testing, the oracle problem, the measurement problem and the impossibility of complete testing. I’d definitely recommend the class as long as you can spare at least 12+ hours a week to commit to watching the videos, reading the required and recommended readings and participating in the assignments. You must take the Foundations course before you can take any further classes so you learn how they are run.

One of my goals for 2013 is to take the other courses: Bug Advocacy, Test Design and then Instructor!