Better Tester Training Materials

Photo by Clark Tibbs on Unsplash

Last month the Association for Software Testing (AST) announced a new partnership with Altom, the owner of BBST®, that enables the AST to refresh our curriculum lineup with the new BBST® Community Track and help fund the future growth of the materials. This partnership and refresh are a huge milestone for the AST. 

For some perspective: Cem Kaner developed the BBST courses over a long period of time, starting with Hung Nguyen in the 90s. In the 2000s after Cem Kaner was recruited to Florida Institute of Technology he and Becky Fielder received grants from the National Science Foundation to adapt them into online courses. Some time later Cem began collaborating with the AST to teach and develop the courses further. Those courses became known as AST-BBST to differentiate the way they developed and were taught (by passionate volunteers). Eventually Cem formed his own company, developed BBST® further and sold it to Altom after he retired. 

BBST® classes are well known for their depth of core testing knowledge and focus on improving through peer review work. That’s great for students but frankly it’s a maintenance challenge. Long before I started with the AST the problem has been, how do we properly maintain and evolve the materials and classes?

In 2018

In 2018, right after the results of our election to the board, Simon Peter Schrijver and I began brainstorming on how we could address some of the long standing issues with AST-BBST. Over the years of teaching we’d heard and experienced a number of complaints including hard to play videos, outdated materials and a difficult to use hosting provider. We felt it was reasonable to address a small subset of problems early on to gain traction. This was our goal:

  • Clean up and rebrand the slides
  • Change system under test for AST-BBST Bug Advocacy
  • Upgrade our teaching platform

In 2020

Slide redesign

Fast forward to 2020 and we’d made significant progress. Simon and I wrote an article with the intent of publishing an update on the progress we were making. Then the Altom partnership happened and things paused.

Until today:


Nearly two years later we are close to making good on those three promises.

The slides for AST-BBST Bug Advocacy and Foundations have been modernized and we’ve gotten rid of all of the old, stale content. We’ve adjusted some lessons and updated a number of the visuals to bring them into the modern age. Meanwhile the same is nearly done for Test Design. We see this as the first steps in hopefully making the static content more readable and consumable. We love the work Cem did, but his slides look like they were put together by a professor (they were!) and aren’t as easy to understand as they could be.

We also updated AST-BBST Bug Advocacy to use LibreOffice as the application under test. With a rich open source community participants get to contribute to the LibreOffice project by helping to review and improve unconfirmed bug reports.

Finally we’ve started work to upgrade our Moodle platform and bring it into a more modern age. We are close to pushing some minor visual updates that allow for better mobile and tablet access. This, however, is a much bigger project that will require more iterations of changes as we figure out how to leverage some of the tools built into the platform such as calendars for easier tracking of due dates.

In keeping with the open source nature of the AST-BBST materials we’ve launched a new site to host and contribute back the changes we are making in our classes. This way anyone who would like to use the new materials created by the AST can. We also think this site might also eventually address a potential self-study version for those who might not want to take an instructor-led version.

Chris Kenst
Simon Peter Schrijver


This article outlines the many improvements the AST-BBST materials have over the original creative commons BBST® materials (which are available at the original Florida Tech site, free for anyone to use). We knew it was a good idea to also keep those updates in creative commons license and give them back to the community, so we:

Importance

The AST/Altom partnership allows BBST® to address 2 distinct customer segments in a more fun and effective way than any certification schemes:

  1. Altom offers full commercial programs classes taught by professional instructors. Students get individualized feedback you’d expect from a commercial course. Corporations looking to offer training to their employees should look here.
  2. The AST offers instructor-supported classes run by our volunteers at an affordable price for most developed countries. Class wide feedback is the norm but you’ll rely more on your peers for review and grading than instructors.

This is all great but what about those materials the AST spent time updating? Well it turns out there’s a 3rd customer segment neither Altom nor the AST serves: Self taught individuals in developing countries. I’m thinking of places like India and the Philippines where even a few hundred USD is far too costly for your average tester.

By making our changes creative commons licensed and public (good forward thinking), this gives the testing community MORE access to great content to learn from and hopefully incorporate into their own teaching.

Better Tester Training Materials

Screenshot from the unfinished Exploratory Testing slides

The AST’s mission is to advance the understanding of the profession and we continue to do this through better tester education. As I said above, this partnership and refresh are a huge milestone but not solely for the AST.

We’ve put a lot of effort into making these materials nice and we encourage you to use them however you see fit. If you are interested in growing yourself or others as software testers, I challenge you to take a look and find a way to leverage the work we’ve done to make your lives better.

Not good enough

An Easy Place to Hide

A month ago someone on LinkedIn thanked a website and the person running it for helping them learn. They recommended others use the site. When people in my network commented on how the site wasn’t any good, I took notice. It reminded me of what Seth Godin said in ‘Not good enough’ is an easy place to hide:

The people who are paying attention are the ones who are trying. And shaming people who are trying because they’re not perfect is a terrific way to discourage them from trying. On the other hand, the core of every system is filled with the status quo, a status quo that isn’t even paying attention.

Seth Godin’s Blog

This is a really hard but important distinction to remember: It’s easy to criticize work in the name of peer review but end up on the bandwagon of not good enough. (There’s a fine line between effective peer review and unwanted comment).

One major lesson I’ve learned from interviewing testers is most aren’t paying attention. They aren’t looking around at how to improve. They don’t read blogs, books or take classes. So while it’s tempting to criticize the work people are putting out, it’s more impactful to reach out to those who are doing nothing and encourage them to try.

How to export environments from Postman

Postman can export data, including collections and environments, to be used outside of Postman. This is especially important when using Newman (their command-line collection runner).

With the release of Postman v8.2 it is easier to find and export Postman collections but harder to find and export environments, hence this article.

(more…)

Hiring a Software Tester, an Analysis

In May of 2020, back when Promenade Group was still called BloomNation, I opened a job posting for a Software Test Engineer. This was to be the first of many test positions we eventually hire for. After going through the whole process of hiring a software tester, I thought it would be useful to analyze the applicant data with the idea of learning something about how I hire and about the applicants who applied.

About the data

Some of this data was collected through our recruiting system and some was manually entered in by me. I spent a good deal of time crunching through raw data in Excel, then coming up with new questions and going back to find more data. Some of the data wasn’t captured at all and so I made guesses / assumptions. Specifically I did this for the applicants location and gender. I don’t hire based on gender, but I was curious to see how this might have effected the final outcome. Despite having 142 submissions, I ended up pulling data on only 107 resumes.

(more…)

2020 in Review

2020 was a year of starts and stops. Of more time but less mental energy. It was a year of developing patience and adapting to hard changes.

Patience

In early March my wife, an ICU nurse here in Los Angeles, saw the first signs of COVID coming in from travelers from Italy. Being ahead of the curve with little ability to do anything other than brace and watch it unfold forced us develop some humility.

In the AST we watched and guessed the trajectory of the virus as it spread country to country derailing our in person meeting. Then it shutdown major conference after major conference. All we could do was wait and see what are options were for our own conference, CAST. By the time it got canceled no one was surprised.

More Time

I spent more time at home with my family and less time on my commute. Despite these benefits I didn’t find more mental energy—quite the contrary. I had plans to attend many virtual conferences and I made it to none of them. Neither free nor paid. I had plans to write for other publications but couldn’t.

Writing became more consistent as I took to putting my frustrations down on paper instead. Yet I hardly published to my blog. I couldn’t get my mind to make space beyond the everyday challenges. There were / are so many things to say but no space available.

(more…)

Hosting a WordPress blog on Digital Ocean

I’ve been running Kenst.com since 2009. While the content has changed over time, the site has remained a blog. Kenst.com has gone from self-hosted on a Windows Home Server in my living room, to Blogger, and ultimately to WordPress. It’s been migrated to and from so many different hosting providers over time that I’ve lost count.

For a time all my sites were hosted on a single “shared” plan. This meant all the sites shared the quasi-dedicated resources of the plan. The problem came when Kenst.com would take the full allocation of memory and crash everything. Even with Cloudflare caching the top pages the traffic was too much. It was time to get a little more serious about hosting.

(more…)

November 2020 Updates

Somehow Thanksgiving is over but I’ve been buying Christmas presents for weeks. It’s amazing how quickly (but not quietly) 2020 is flying by now that it is nearly December. As with most of my updates, these are mostly for my own clarity on what has transpired but I hope they are of interest to bystanders and friends too.

About two months ago my company BloomNation announced it was rebranding itself to Promenade Group to better reflect how our business was now positioned to help 3 different verticals gain traction online: flower shops (BloomNation), liquor and wine stores (Swigg) and restaurants (Dig In). The company is growing rapidly and is in a strong position during Pandemic (thankfully). Most companies I’ve joined have been duds so working for a startup that is succeeding is a nice change of pace.

A few months prior to this, I hired my first direct report. I spent at least a full month recruiting and interviewing people for a mid-level Software Test Engineer. I’ll have more to say in the near future but suffice it to say over 100 people applied and only 1 got the job.

So many other things come to mind:

  • A little over 2 weeks ago was my first AST Board Meeting as President. I spent the past two years as Treasurer, et al. Not much has changed, but then again so much has.
  • There’s lots of cool stuff happening within the AST that I hope to be able to share in the next few months! So much good stuff. In the meantime I’ve got to bite my tongue.
  • I’m currently teaching the final AST-BBST course of the year, Test Design. One of the most under-rated courses we have. Lots to look forward to in 2021 with regard to courses!

While I’m not speaking at it, the original online testing conference (the aptly named) OnlineTestConf is coming in December is always worth checking out. The 2020 conference calendar isn’t over but has been filled with online conferences. If this is the way of the future, we’ll definitely need better ways to comb through the programs to find the specific things we want to learn. Information overload is real.

I’m Speaking at TestFlix

It’s true, I’m speaking at TestFlix on November 28th, 2020. You should sign up to join; it’s free to register!

I recorded and submitted my 7 minute talk on “Using Test Idea Catalogs for Better Testing”. The premise is:

Testers can develop a set of tests or test concepts for a specific object or risk and re-use them in similar projects or products. Catalogs come in many shapes and forms, they can be lists or more detailed. They can be public or private. They can be developed by individual testers or as teams within companies. But they all help you test better!

Slides

References

I plan to write more on the topic of Test Idea Catalogs in the near future but I mentioned a few in the presentation that I’d like to call out here:

Upgrading to WebDriverIO 6

In March I went through the process of upgrading to WebDriver v5. Last month I took the next step by upgrading our deployment to version 6 so we’d be current. I learned quite a bit from that first upgrade which made this upgrade a whole lot easier.

Here’s what I did to upgrade to WebDriverIO 6:

  1. Checked the Docs. I scanned through the change log sections on breaking changes. Maybe I’m boring but nothing in the docs breaking changes notes looked to impact our setup! 👏
  2. npm oudated . This is one of my favorite (and mostly unused) npm commands. It told me specifically what packages I had that were outdated. Hint: it was all of them. 🤯
  3. Upgrade the easy stuff.
    1. Based on what npm returned, I began by updating my package.json file for 3rd party libraries. Libraries such as prettier, chromedriver, moment, etc. that all played a part in my solution but didn’t deal directly with my tests. Then I npm install the latest changes.
  4. Run the whole test suite looking for failures.
  5. Upgrade the harder stuff.
    1. Same thing as step 3, updated the package.json versions to be the “latest” and then npm install.
    2. Another way to do this is to remove the node_modules folder and re-install each package based on the install instructions.
  6. Run the whole test suite… until nothing is broken!

More reasonable!

This upgrade wasn’t as daunting as I initially feared. With upgrades there’s always some level of concern you are going to “ruin a good thing”. Once I moved past that concern, there wasn’t much effort involved in getting things to work. Lots of credit goes to the WebDriverIO team for making the process more reasonable and straightforward, which of course they stated in their announcement:

This major update is much more reasonable and contains subtle changes that will help the project further grow while remaining performant at the same time.

I’ve been pretty happy thus far using WebDriverIO and I’m excited to see where things go!

The TestOpsy

Back in January I hosted James Bach and Michael Bolton for an AST webinar on the concept of a TestOpsy or a way to learn about the testing you do by dissecting it. Below you’ll find not only a description and the webinar video but a transcription for what I hope is easier reading.

By looking very carefully at what you actually do, identifying your own heuristics, and putting that process into descriptive, evocative words, you can discover surprising depths in each act of testing you perform. In a testopsy, you build your skills of observation, narration, and test framing. You may even discover a technique no one yet has written about. And if you do it with a colleague, it stimulates discussion on test design.

James Bach & Michael Bolton

What do we mean by a Testopsy

We are talking about an autopsy for testing. We are talking about taking a very close look at a session of testing and you can do a testopsy based on just a few minutes of testing.

(more…)