My First Term on the AST Board of Directors

The Association for Software Testing (AST), a non-profit professional organization dedicated to advancing the understanding and science of software testing, has announced a call for nominations for the Board of Directors for 2020-2022. This means my two-year term as a director is coming to an end. I feel fortunate and grateful to announce I’m running for a second term. The AST has helped a lot of people including me. For this and a few other reasons described below, it feels like the right moment to reflect on what it has been like to help run this global non-profit.

The Golden Ticket

I was elected in August of 2018 while attending the Conference for the AST (CAST) in Melbourne, Florida. An AST member since 2012, I started volunteering in 2013 after I became an AST-BBST Instructor. Coming up through BBST, I thought educational advocacy was one of the AST’s most important community services. You can’t advance the understanding of the craft until testers have a solid understanding of what already exists. I really wanted to improve our offering and felt the best way was to help set priorities at the board level. 

AST Board of Directors

Elections happen every year with roughly half of the 7 person board up for election each year. The election process starts with a call for nominations and then candidates introduce themselves via questions posted to the web. Finally voting takes place during the time of CAST (typically the first week of August) and on the final day of the conference a new board is announced.

As a member-elect you are elected to a position by the sitting board members based, in part, on your preferences. In 2018 during a discussion with existing board members it came up there was a need for someone to take on the Treasurer position. It wasn’t the role I initially wanted (VP of Education was my first choice) but I felt reasonably competent so I accepted.

As with any official board position it’s a starting point for your contributions. I really wanted to focus on education but my fellow AST-BBST instructor Simon Peter (with whom I taught countless classes) wanted the position as well. We quickly both decided it made sense for him to take VP of Education and I take Treasurer. Just like we had done in our teaching we decided it would be fun to collaborate on the many changes we wanted to see in AST-BBST. I had my official role, Simon had his and yet we worked together whenever we could to improve our educational program.

(more…)

FAQs to How I Became An Automation Engineer (talk)

I’ve given the talk on How I Became an Automation Engineer a few times. Each time I’ve gotten good feedback and a lot of questions from engaged participants. With each repeated question, I try to update my talk to address those points for future audiences. However often I do this some repeated questions still arise. I’ve taken this challenge as both an excuse to write out my answers and try out a FAQ schema block!

Frequently Asked Questions

Here are the FAQs for my talk How I Became an Automation Engineer.

How do you decide the scripting or programming language to use in your testing? Do you typically use the same language the app is built in?

My recommendation is to use a programming language that your company supports. By this I mean a language that someone within your company uses and can provide feedback on your code through code reviews. This may or may not be the language the application is built in. Most companies use several languages. You might build your tools in JavaScript while the application is built in Java.

Ultimately your goal is to tackle the task at hand. Use the tools and languages that make sense.

Is (insert name of your current tool) sufficient enough to perform all types of automation testing?

No, probably not. Hopefully you work for an organization that wants to use the right tool for the right job.

Part of the process for learning about tools is we figure out what they are and aren’t good for. No one tool is good for all jobs. The same goes for the person using the tool. If you only know one tool you’ll try to use it for every problem. The more tools we learn, the more likely we will use the right tool for the right job. This also makes us more valuable in the market for new jobs.

What are your thoughts on using frameworks with record features to start learning automation?

It depends on your needs and your skills.

In the past, Record and Playback features were sold as Snake Oil. They’d fix all your problems. But they usually were much good beyond proof of concepts.

Today, more and more services are cropping up with codeless / record and playback capabilities. If these meet the needs of your project and/or company, then use them. One challenge for the individuals learning and using these these tools is they might not benefit your (market) value in the long term.

Sometimes the team or project doesn’t support automation either for budget or time concerns. How do you tackle that?

Test Automation is there to provide feedback, but it costs money and takes time to build and maintain. So if you are on a short term project then it might not be worth the time an effort to do certain kinds of test automation.

What kinds make sense? Well it depends on the project. It might be worth it to do unit tests but not much else. Or it might be worth it to build some one-off probabilistic automated tests to help you discover problems because it doesn’t have to be maintained. Remember the point of automation is to reduce our time or extend our capabilities but all that must be done within the context of your project.

Do you recommend learning to test and automate at the API level? Is Postman adequate for this?

Yes! In general I recommend automating tests at the unit level first, then at the service or API level and lastly at the UI level. Learning to explore APIs is a big part of this and eventually once you explore them, you can create automated tests.

Postman is a good tool for exploring APIs (there are probably others). It might also make sense for you to write some automated tests in Postman but that really depends on your goals and/or if there are better places to do it. This depends on your overall strategy.

According to your last slide, the future of the Automation Engineer role is uncertain. Do you think it’s better to move to the developer role or there is still hope that automation roles will be still exist?

We know that in high performing teams developers own the automated tests and testers are allowed to explore for risks. So there’s a high probability in companies looking to build high performing teams that Automation Engineering roles will disappear or not exist. There’s also a high probability of developer roles being needed in the future. We just don’t know when any of this will occur. If you are going to move, it makes sense to survey the landscape and see what the options are, see what your skill levels are and be practical.

What is the name of the book about you recommend / quote about high performing teams handling test automation?

Want to watch the latest recording of this talk?

Five for Friday – April 17, 2020

Welcome to Friday, here are five points worth sharing. A few of these are “work” related but not all. Here is to helping find balance between working from home, home time and personal time:

  • I’m finishing my second week of One Hundred Push ups. It’s hard but fun and takes less than 10 minutes per day. I’ve hit a few walls (because I’m out of shape) but the next workout (I do 3x a week) I usually break down the wall and continue on.
  • Arnold Schwarzenegger has been busy on social media helping people focus on what they can control. Then he shared his old no gym / home workout routine using just bodyweight. It’s kind of legit. Men’s Health has it. I’ll try it after I can do 100 push ups!
  • For those parents who are looking for ways to plan their children’s education while being remote we just signed up for ABCmouse.com for our 2.5 year old. Between this and the iOS app Endless Learning Academy I think it’s going well.
    • We’ve actively been trying to experiment with learning plans to keep the kid developing. He doesn’t go to preschool yet but then again it seems like no kids currently do.
  • Tea-Time with Testers magazine has a new issue after over 2 years on hiatus. Included was an article that struck home with me called How To Read A Difficult Book by Klára Jánová and James Bach. I also have tried reading An Introduction To General Systems thinking several times and have not gotten what I wanted from it. Perhaps I need to try what Klára did.
  • There are some good bits of advice in the a16z podcast: Moving to Remote Development (and work). One is about the kinds of bad behaviors that get emphasized when moving to remote work (especially for managers). Another is a discussion around communication, using video, how to be expressive and generally send the right signals.

And a friendly reminder that with Amazon delivering slower than normal for non-essential things, eBay sellers are shipping faster than ever!

Upgrading to WebDriverIO 5

A few weeks ago I finished upgrading our implementation of WebDriverIO from version 4 to version 5. The impetus for the upgrade was an announcement by the WebDriverIO twitter account of a new beta version 6 to be quickly followed by a finished version (it’s already here). One thing was clear: you have to be on v5 to go to v6 and each new subsequent version would only be supported for a year. Time to upgrade!

I’d been on version 4 since I originally deployed WebDriverIO in mid 2018. I knew version 5 was out but I had no immediate plans to upgrade given all of the warnings around breaking changes.

Preparation

This isn’t to say I wasn’t preparing myself. I created a JIRA ticket to outline what the work might look like. I was going through the TestAutomationU course on WebDriverIO which uses V5 and of course practicing in my own repo. With confirmation of the EOL of version 4 it was time to move on. 

I scoped out more of the work in JIRA (and also made a story to upgrade to v6). I bookmarked a few important articles including the WebDriverIO blog post announcing version 5 which highlights, among other things, specifically How to Upgrade. Then I made note of the methods that were either changing or being replaced for easier reference.

Upgrading to WebDriverIO 5

Finally it was time for the upgrade itself. The first question I needed to answer was logistical: how do I approach making the changes? Create a whole new repo, install the new packages and then move my tests over? Or just upgrade in place? Despite the daunting feeling I had, I figured it would be easier to upgrade in place and deal with the test failing as they came. This would introduce less changes than actually trying to move things over. Then it was time to follow the recommendations in the blog post:

  1. Remove node_module
  2. Remove all wdio-* packages from package.json
  3. Install the latest version of webdriverio: $ npm install [email protected]
    1. Note: if you did this today, you’d want to npm install [email protected] however I’d recommend going straight to v6 instead of going to v5 and then v6.
  4. Install the new wdio testrunner: $ npm install @wdio/cli --save-dev
  5. I have multiple configuration files and so I didn’t need to back them up. Rather I created a new one as part of the webdriverio configuration wizard. Then I eventually migrated / pruned the original ones until they were what I wanted.
  6. Rerun the configuration wizard: $ npx wdio config

All the Broken Things

Bam! WebDriverIO 5 deployed. Kind of. The easy part was done, next up was running each test one at a time, starting with the easiest / shortest tests. (I usually start with low hanging fruit so that I build momentum). When a test failed, I’d find out where and why (due to a rename or deprecation) and make a change. Rise and repeat for the whole test suite.

If this sounds simple, that’s because it was. All it took was time to remember where things were and why something was done a certain way and then make a change. 90% of the time this worked fine.

Other times it was more complicated. In at least one instance I ran into a breaking change, the functionality that was there before didn’t exist in the new version. I commented out 2 tests and moved on. (Speaking of which, I need to open a bug report about this).

Benefits

While it’s never fun to take something that’s working and break it just to upgrade, it does have some side benefits. Such as…. I fixed a few remaining bad ternary statements. Then cleaned up some unused libraries / plugins that I installed for some reason but can’t possibly remember anymore. Also refactored a bit of my code to make things simpler. When I was going through the TestAutomationU course I got to see how the author structured her tests. Now I have a new task to break apart my larger tests into smaller more defined tests with less assertions. 

All of this is to say, once you start making changes to improve one thing, it can snowball and lead to even more improvements. 

If you liked this article consider sharing it or buying me a coffee:

Regression testing isn’t only about repetition

Often when I’m chatting with someone about their regression testing strategy there is an assumption regression is all about repeating the same tests. This is a bit problematic because it ignores an important aspect which testers tend to be good at: focusing on risk. A better way to think of regression testing is it can be applied in two different ways: Procedurally and Risk-Focused

Procedural Regression Tests

When I speak of procedural I mean a sequence of actions or steps followed in regular order.  As I said above this seems to be the primary way people think about regression testing: repetition of the same tests. This extends to the way we think about automating tests as well.

Procedural regression testing can be quite valuable (so far as any single technique can be). The most valuable procedural regression tests are unit tests when applied to our CI system and run regularly. In this way they become a predictable detector of change, which is often why we run regression tests in the first place. (Funny enough automated UI tests are some of the most common procedural regression tests but aren’t the best detectors of change). 

The big problem with procedural regression tests are that once an application has passed a test, there is a very low probability of it finding another bug. 

Risk-Focused Regression Tests

When I speak of risk-focused I mean testing for the same risks (ways the application might fail) but changing up the individual tests we run. We might create new tests, combine previous tests, alter underlying data or infrastructure to yield new and interesting results.

To increase the probability of finding new bugs we start testing for side effects of the change(s) rather than going for repetition. The most valuable risk focused regression tests are typically done by the individual testers (or developers) who know how to alter their behavior with each pass through the system.  

A Combined Approach

Thinking about regression testing in terms of procedural and risk focus allows us to see two complementary approaches that can yield value at different times in our projects. It also gives testers an escape from the burden that comes  with repetition while still allowing us to meet our goals.

On other platforms:

If you liked this article please consider sharing it or buying me a coffee:

I like working from home… 🏠

I like working from home… but it feels different now. 

Clarification: I like working from home, when it’s my choice.

My ideal work/life choice would involve: splitting the week into 2 or 3 days of in-office and from-home work. The extrovert in me would get to talk to my coworkers, pair up and communicate freely. The introvert in me would get to focus with limited distractions, kill my commute time and get to see my family a little more.

Back in the day I worked fully remote for over 3 years straight but this time I no longer have the same choices to keep me balanced. My old office became the kid’s bedroom and although I love my standing desk it’s also in the dining room. Staying inside for most of the workweek used to be fine when I could go out to eat for lunch or dinner or spend time with friends. Those options are much reduced today.

However even as we retreat from physical interaction I’m reminded there’s still the possibility to learn and connect with others. This morning I was Skyping with a friend in the Netherlands who had just gotten off of a random Zoom call with 30+ Strangers for an happy hour drink. Someone posted a link, others joined and enjoyed an after work drink together!

And that’s the point. There are still choices to be made despite our shelter-in-place orders. I take long walks with the family after work and on occasion I fly my drone to explore my surroundings. This weeks goal is to start taking walking breaks at regular interviews where I’ll listen to my podcasts and perhaps say hi to people doing the same. Who knows, maybe I’ll host or join social hangouts for fun?

Five for Friday – March 13, 2020

Welcome to Friday, here are five points worth exploring:

February 2020 Updates

This month has been busier than the last few and I’m feeling the need to chill. Part of this busyness is having just wrapped up teaching an AST-BBST Foundations class, which is about equally as intense for the instructors as they are for the students. The other part is at work, we are through our biggest week of the year aka Valentines Day (did you order your flowers?). Oh and my 49ers lost the Super Bowl. It’s fine, everything is fine now. At least I’ve settled in with my team being second best in the league this year. 

Come to think of it there are a few more things happened that are worth mentioning:

  • On March 20, 2020 I’ll be speaking at a new free online conference by Kobiton called Odyssey on the topic of How I Became A Test Automation Engineer. This will be an updated (read better!) version of the talk I gave at the OTC. The conference takes place March 19-20, 2020 and you can sign up for free!
  • I’m now the proud owner and maintainer of the Context-Driven-Testing website. This is the official CDT website created by Cem Kaner to share the CDT principles. Along with the principles comes a blog that I hope to add to over time. This is a huge thing for me as someone who takes a CDT approach to testing, to have this piece of history.
  • In some ways I consider myself a bit of a historian of the testing community, especially around the areas of the community where “I came up”, e.g. around CDT. If you saw last month’s update you’ll know I also manage the LAWST website, which although is about Peer Workshops, definitely comes from the CDT community. To me these are some prized possessions because they contain a lot of history, a lot of writings about the growth and challenges of software testing community in general.
  • It’s crazy to me that most of TestingConferences.org’s Sponsorships for the year 2020 are gone! I’m still working on a way to get Sponsorships set up here, but that’s on the long list of items in my backlog. Maybe next month I’ll get the time to do this?