Better Tester Training Materials

Photo by Clark Tibbs on Unsplash

Last month the Association for Software Testing (AST) announced a new partnership with Altom, the owner of BBST®, that enables the AST to refresh our curriculum lineup with the new BBST® Community Track and help fund the future growth of the materials. This partnership and refresh are a huge milestone for the AST. 

For some perspective: Cem Kaner developed the BBST courses over a long period of time, starting with Hung Nguyen in the 90s. In the 2000s after Cem Kaner was recruited to Florida Institute of Technology he and Becky Fielder received grants from the National Science Foundation to adapt them into online courses. Some time later Cem began collaborating with the AST to teach and develop the courses further. Those courses became known as AST-BBST to differentiate the way they developed and were taught (by passionate volunteers). Eventually Cem formed his own company, developed BBST® further and sold it to Altom after he retired. 

BBST® classes are well known for their depth of core testing knowledge and focus on improving through peer review work. That’s great for students but frankly it’s a maintenance challenge. Long before I started with the AST the problem has been, how do we properly maintain and evolve the materials and classes?

In 2018

In 2018, right after the results of our election to the board, Simon Peter Schrijver and I began brainstorming on how we could address some of the long standing issues with AST-BBST. Over the years of teaching we’d heard and experienced a number of complaints including hard to play videos, outdated materials and a difficult to use hosting provider. We felt it was reasonable to address a small subset of problems early on to gain traction. This was our goal:

  • Clean up and rebrand the slides
  • Change system under test for AST-BBST Bug Advocacy
  • Upgrade our teaching platform

In 2020

Slide redesign

Fast forward to 2020 and we’d made significant progress. Simon and I wrote an article with the intent of publishing an update on the progress we were making. Then the Altom partnership happened and things paused.

Until today:


Nearly two years later we are close to making good on those three promises.

The slides for AST-BBST Bug Advocacy and Foundations have been modernized and we’ve gotten rid of all of the old, stale content. We’ve adjusted some lessons and updated a number of the visuals to bring them into the modern age. Meanwhile the same is nearly done for Test Design. We see this as the first steps in hopefully making the static content more readable and consumable. We love the work Cem did, but his slides look like they were put together by a professor (they were!) and aren’t as easy to understand as they could be.

We also updated AST-BBST Bug Advocacy to use LibreOffice as the application under test. With a rich open source community participants get to contribute to the LibreOffice project by helping to review and improve unconfirmed bug reports.

Finally we’ve started work to upgrade our Moodle platform and bring it into a more modern age. We are close to pushing some minor visual updates that allow for better mobile and tablet access. This, however, is a much bigger project that will require more iterations of changes as we figure out how to leverage some of the tools built into the platform such as calendars for easier tracking of due dates.

In keeping with the open source nature of the AST-BBST materials we’ve launched a new site to host and contribute back the changes we are making in our classes. This way anyone who would like to use the new materials created by the AST can. We also think this site might also eventually address a potential self-study version for those who might not want to take an instructor-led version.

Chris Kenst
Simon Peter Schrijver


This article outlines the many improvements the AST-BBST materials have over the original creative commons BBST® materials (which are available at the original Florida Tech site, free for anyone to use). We knew it was a good idea to also keep those updates in creative commons license and give them back to the community, so we:

Importance

The AST/Altom partnership allows BBST® to address 2 distinct customer segments in a more fun and effective way than any certification schemes:

  1. Altom offers full commercial programs classes taught by professional instructors. Students get individualized feedback you’d expect from a commercial course. Corporations looking to offer training to their employees should look here.
  2. The AST offers instructor-supported classes run by our volunteers at an affordable price for most developed countries. Class wide feedback is the norm but you’ll rely more on your peers for review and grading than instructors.

This is all great but what about those materials the AST spent time updating? Well it turns out there’s a 3rd customer segment neither Altom nor the AST serves: Self taught individuals in developing countries. I’m thinking of places like India and the Philippines where even a few hundred USD is far too costly for your average tester.

By making our changes creative commons licensed and public (good forward thinking), this gives the testing community MORE access to great content to learn from and hopefully incorporate into their own teaching.

Better Tester Training Materials

Screenshot from the unfinished Exploratory Testing slides

The AST’s mission is to advance the understanding of the profession and we continue to do this through better tester education. As I said above, this partnership and refresh are a huge milestone but not solely for the AST.

We’ve put a lot of effort into making these materials nice and we encourage you to use them however you see fit. If you are interested in growing yourself or others as software testers, I challenge you to take a look and find a way to leverage the work we’ve done to make your lives better.

Hiring a Software Tester, an Analysis

In May of 2020, back when Promenade Group was still called BloomNation, I opened a job posting for a Software Test Engineer. This was to be the first of many test positions we eventually hire for. After going through the whole process of hiring a software tester, I thought it would be useful to analyze the applicant data with the idea of learning something about how I hire and about the applicants who applied.

About the data

Some of this data was collected through our recruiting system and some was manually entered in by me. I spent a good deal of time crunching through raw data in Excel, then coming up with new questions and going back to find more data. Some of the data wasn’t captured at all and so I made guesses / assumptions. Specifically I did this for the applicants location and gender. I don’t hire based on gender, but I was curious to see how this might have effected the final outcome. Despite having 142 submissions, I ended up pulling data on only 107 resumes.

(more…)

I’m Speaking at TestFlix

It’s true, I’m speaking at TestFlix on November 28th, 2020. You should sign up to join; it’s free to register!

I recorded and submitted my 7 minute talk on “Using Test Idea Catalogs for Better Testing”. The premise is:

Testers can develop a set of tests or test concepts for a specific object or risk and re-use them in similar projects or products. Catalogs come in many shapes and forms, they can be lists or more detailed. They can be public or private. They can be developed by individual testers or as teams within companies. But they all help you test better!

Slides

References

I plan to write more on the topic of Test Idea Catalogs in the near future but I mentioned a few in the presentation that I’d like to call out here:

The TestOpsy

Back in January I hosted James Bach and Michael Bolton for an AST webinar on the concept of a TestOpsy or a way to learn about the testing you do by dissecting it. Below you’ll find not only a description and the webinar video but a transcription for what I hope is easier reading.

By looking very carefully at what you actually do, identifying your own heuristics, and putting that process into descriptive, evocative words, you can discover surprising depths in each act of testing you perform. In a testopsy, you build your skills of observation, narration, and test framing. You may even discover a technique no one yet has written about. And if you do it with a colleague, it stimulates discussion on test design.

James Bach & Michael Bolton

What do we mean by a Testopsy

We are talking about an autopsy for testing. We are talking about taking a very close look at a session of testing and you can do a testopsy based on just a few minutes of testing.

(more…)

I’m running for the 2020-2022 AST Board of Directors

Elections just opened for the Association for Software Testing’s Board of Directors for which I’m a candidate. If you are a voting-eligible member of the AST I’d appreciate the consideration as I run for my 2nd term.

For those who are voting (or possibly just interested) I completed a list of candidate questions. For fun you can compare them to my answers from 2018.

I did want to highlight a few questions that I think are important:

(more…)

My First Term on the AST Board of Directors

The Association for Software Testing (AST), a non-profit professional organization dedicated to advancing the understanding and science of software testing, has announced a call for nominations for the Board of Directors for 2020-2022. This means my two-year term as a director is coming to an end. I feel fortunate and grateful to announce I’m running for a second term. The AST has helped a lot of people including me. For this and a few other reasons described below, it feels like the right moment to reflect on what it has been like to help run this global non-profit.

The Golden Ticket

I was elected in August of 2018 while attending the Conference for the AST (CAST) in Melbourne, Florida. An AST member since 2012, I started volunteering in 2013 after I became an AST-BBST Instructor. Coming up through BBST, I thought educational advocacy was one of the AST’s most important community services. You can’t advance the understanding of the craft until testers have a solid understanding of what already exists. I really wanted to improve our offering and felt the best way was to help set priorities at the board level. 

AST Board of Directors

Elections happen every year with roughly half of the 7 person board up for election each year. The election process starts with a call for nominations and then candidates introduce themselves via questions posted to the web. Finally voting takes place during the time of CAST (typically the first week of August) and on the final day of the conference a new board is announced.

As a member-elect you are elected to a position by the sitting board members based, in part, on your preferences. In 2018 during a discussion with existing board members it came up there was a need for someone to take on the Treasurer position. It wasn’t the role I initially wanted (VP of Education was my first choice) but I felt reasonably competent so I accepted.

As with any official board position it’s a starting point for your contributions. I really wanted to focus on education but my fellow AST-BBST instructor Simon Peter (with whom I taught countless classes) wanted the position as well. We quickly both decided it made sense for him to take VP of Education and I take Treasurer. Just like we had done in our teaching we decided it would be fun to collaborate on the many changes we wanted to see in AST-BBST. I had my official role, Simon had his and yet we worked together whenever we could to improve our educational program.

(more…)

Regression testing isn’t only about repetition

Often when I’m chatting with someone about their regression testing strategy there is an assumption regression is all about repeating the same tests. This is a bit problematic because it ignores an important aspect which testers tend to be good at: focusing on risk. A better way to think of regression testing is it can be applied in two different ways: Procedurally and Risk-Focused

Procedural Regression Tests

When I speak of procedural I mean a sequence of actions or steps followed in regular order.  As I said above this seems to be the primary way people think about regression testing: repetition of the same tests. This extends to the way we think about automating tests as well.

Procedural regression testing can be quite valuable (so far as any single technique can be). The most valuable procedural regression tests are unit tests when applied to our CI system and run regularly. In this way they become a predictable detector of change, which is often why we run regression tests in the first place. (Funny enough automated UI tests are some of the most common procedural regression tests but aren’t the best detectors of change). 

The big problem with procedural regression tests are that once an application has passed a test, there is a very low probability of it finding another bug. 

Risk-Focused Regression Tests

When I speak of risk-focused I mean testing for the same risks (ways the application might fail) but changing up the individual tests we run. We might create new tests, combine previous tests, alter underlying data or infrastructure to yield new and interesting results.

To increase the probability of finding new bugs we start testing for side effects of the change(s) rather than going for repetition. The most valuable risk focused regression tests are typically done by the individual testers (or developers) who know how to alter their behavior with each pass through the system.  

A Combined Approach

Thinking about regression testing in terms of procedural and risk focus allows us to see two complementary approaches that can yield value at different times in our projects. It also gives testers an escape from the burden that comes  with repetition while still allowing us to meet our goals.

On other platforms:

If you liked this article please consider sharing it or buying me a coffee:

2019 in Review

My yearly tradition has been to summarize the most popular and important (to me) articles I’ve written over the past year along with some reflections and other forward-looking (and likely wrong) statements mixed in.

You can find previous years in review here: 2018 | 2017 | 2016 | 2015

Santa Monica Pier, near my office

What happened in 2019?

I joined BloomNation as a Test Automation Engineer in 2018 and in 2019 was promoted to senior role. The promotion was in part my testing contributions and hitting my goals but also the general impact I’ve had on the company outside of that role. During CAST I was telling people I had temporarily switched positions by taking on a Product Manager role until we could fill the vacancy. While temporary, I helped the company continue to deliver on a new core piece of the business and in turn it gave me some new experience and perspective. I plan to write about this experience soon. 

Speaking of CAST 2019, this was my first year helping to organize and run a conference of any kind. I plan to write about this experience as well but suffice it to say this was both fascinating and incredibly hard. 

I did some speaking in 2019, made my first and second podcast appearance and found what I hope is a sustainable model for supporting TestingConferences.org. I’ve started to become more serious about potential ways to support the things I do. In fact one theme of 2020 might be me figuring out how to balance all the things I want to do with all the things I’ve already committed to doing.  Just like every year!

The Five Most-Viewed Articles: 

  • How to debug problems on Mobile Safari – After showing a co-worker how they could debug mobile safari problems on their MacBook, I realized it wasn’t common knowledge. So I helped change that a bit but making it clear how to do it.
  • Participating in Code Reviews as a Tester – I’ve always liked the concept of helping testers push their technical understanding to reduce both risk and increase confidence. This post was based on a webinar of the same name and includes the slides and a link to watch the webinar.
  • How I Became an Automation Engineer – A talk based on my personal experiences of becoming an Automation Engineer and what my role looks like. This also became a blog post with references, slides and an embedded video.
  • Move Fast and Make Things Better – I much prefer the saying move fast and make things better over move fast and break things and this article points out why.
  • It’s Easier to Write about Tooling – Whenever I go to conferences there is a heavy emphasis on what our tooling is. Even in my own writing there can be a heavy emphasis on tools and I think that’s just because it’s easier than writing about the decisions we took and models we made prior to choosing it.

Over 185k page-views in 2019

The first article made it into my top 10 articles over all, which is great. Traffic to this site continues moving up and to the right over time. In 2019 alone I had more than 185k page-views. That’s double the views in one year! Wow!

A Few Other Articles

  • It was the Creative Web that collapsed – The title of this post is a line from Edward Snowden’s new book, Permanent Record. It’s a great book, you should read it. I reflected on how Surveillance Capitalism is part of the world of the web and how I hope to limit that surveillance on all my sites.
  • Getting Paid to Learn was a reminder of how important it is to have the company you work for support your professional development. Making promises is one thing but taking action to do it has a positive effect on the people and the company culture.
  • You Don’t Have to be an Expert to Teach is a friendly reminder that all we need to do to teach someone something is know slightly more than they do.

These are articles were written because I was feeling the flow and when inspiration hits you take it. They turned out well, I’m proud to write consistently about topics that intrigue me and I hope there’s intrigue on the other side too. 

The Future is already here

Predicting the future is fun and yet meaningless. But here are the things already on my radar for 2020. So much to consider and so little time:

  • CAST 2020 is already rolling with early bird ticket sales open.
  • I’m almost done teaching an AST-BBST Foundations course, my first in a few years and I’m constantly looking at the course with an eye for improvements.
  • Speaking of improvements, I’m still helping to redesign some aspects of the course AND of course trying to be a treasurer and handle all the financial things.
  • When it comes to writing I hope to continue the pace of 2x blog posts per month. It’s challenging but doable. More would be great but I want a constant pace to keep myself going.
  • I also hope to start publishing / writing more articles that touch on JavaScript, CypressIO and WebDriverIO. I’ve been primarily using JavaScript in the last year in a half and I’ve written almost no examples using it.
  • Despite record setting viewership in 2019, I’m hoping to get better at self branding and sharing which should see page-views increase YOY again. I’ve already updated the newsletter to be less often so I can focus on driving readership for certain articles. I also intend to cross-promote on other blogging and reading platforms to gain viewership. So much to do.
  • I essentially did 3 presentations / talks in 2019 and I intend to do the same or a few more this year (especially for the online conferences). Hopefully I’ll also get around to making a few more podcast appearances because they are fun to do and fun to share.

Cheers to the rest of 2020! What will you be doing?