Five for Friday – July 26, 2019

I have now managed to keep my publishing trend of 4 weeks straight with this post! Here are five links worth exploring:

Alright this is slightly more than five links worth exploring but you get the idea. Did I miss anything important?

Do things that don’t scale

If you can find someone with a problem that needs solving and you can solve it manually, go ahead and do that for as long as you can, and then gradually automate the bottlenecks. – Paul Graham

With the title of Automation Engineer it might seem like automating bottlenecks is “my jam” and therefore “do[ing] things that don’t scale” might seem kind of an odd thing for me to agree with. However it turns out this is a useful way to think through solutions to problems, including testing problems.

From a business standpoint this makes a lot of sense, why waste time automating something until you are sure it has value? Many startups have blown through money building solutions to things (or automating problems) that aren’t actually important (or real problems). If you are an established company and thinking of branching into a new market segment, it’s ok to do everything by hand at first: signing up customers, placing orders, building website, etc.

From a software standpoint this also makes a lot of sense. Don’t worry about automating bottlenecks like test setup or execution until:

  1. You are sure the tests have value (business or technical) AND
  2. They’ve become bottlenecks to delivery

Otherwise don’t waste your time.

Participating in Code Reviews as a Tester

Have you ever wondered what a code review is and/or what it’s like to participate in one? Are you a tester, product or business person who regularly interacts with the output of the code but wonders if they could catch bugs earlier by “shifting left”?

Turns out you don’t have to know how to code in order to learn about the changes, question the assumptions, clarify ambiguities, and generally participate in the knowledge-sharing around Code Reviews. Here’s a presentation on how to get started and what to look for.

Simply register to gain access, and the video will start right away.

Slides

References

Move fast and make things better

I much prefer the saying move fast and make things better over move fast and break things. The latter might be more popular but the former is more realistic.

Moving fast (software agility) used to be a business advantage but now, at least for any service business, it’s mostly a requirement. Fast implies some things might “break” and while that’s part of the process it should never be our aim. (We should have flexible systems in place to help us quickly identify and fix those “breaks”. Moreover a DevOps culture can help us improve stability so “breaks” are less severe.)

A more realistic mission should be to solve our customers problems and do so in a way that continually improves. Where failure doesn’t mean we stop but rather we iterate until it’s right. This way we solve the customers problems while being able a benefit to our business.

Now let’s go move fast and make things better!!

Join my Webinar on Participating in Code Reviews as Tester

Last year I wrote an article for Stickyminds about Participating in Code Reviews as a Tester where I made the case for Code Reviews being more than just a chance to catch bugs. They also serve as a chance to see how something is built and have a conversation about it.

We, as Testers, question software differently from developers so it’s important that we participate in this knowledge-sharing practice and now I’m going to try to show everyone how!

I’ve partnered with TestCraft for a webinar on May 21, 2019 at 9am PST.

I will demonstrate a live Code Review (or two) and show everyone it’s not about knowing how to code, it’s about asking questions and learning about the changes.

Expect to Learn:

  • The basic git workflow, where the creation of a Pull Request leads to Code Reviews
  • What to look for when you are Code Reviewing from a tester’s perspective
  • A practical way to get started doing your Code Reviews on your own
    The webinar will end with a Q&A.

[button url=”https://www.testcraft.io/webinar-participating-code-reviews/?utm_campaign=Participating%20in%20code%20reviews%20as%20a%20tester%20webinar%20-%20chris%20link&utm_source=linkedin&utm_medium=social” color=”blue” size=”medium” type=”square” target=”_blank”] Sign Up Here! [/button]

late-April Updates

April has come and is nearly gone without any prose being published. I couldn’t have that. You see I’ve been writing but I haven’t condensed that prose into a nice enough package to share. In the meantime lots of things are happening that are worthy of sharing:

    • On Friday, May 3rd at 10:00am PST, I’ll be hosting Doug Hoffman for the AST’s May webinar on “The Often Overlooked Test Oracle: The Key to More Powerful Testing”. I love talking with Doug because he’s so knowledgeable and frankly I think Test Oracles aren’t very well understood. Learn More or Sign Up.
    • Speaking of the AST, our conference CAST 2019 is open for early bird registration AND all the speakers have been announced. I’m already signed up! You should too.
    • Despite being the “just” the Treasurer for the AST, I’m actively involved in most aspects of the business including finance, marketing, SEO, IT, conferences, education and elections to name a few. Ultimately what this means is I spend a good deal of my time trying to help improve the organization. It’s a fun challenge.
    • I’ll be doing a half-day tutorial at StarWest 2019 on “Testing Today’s Web Applications: Tools You Can Use” which I’m super excited about.
    • I’ve agreed (and am excited) to be doing an online conference talk on my experience of being an Automation Engineer. More details to come.
    • I made an appearance on the AB Testing Podcast’s 100th episode two weeks ago. It was a lot of fun and honestly it’s my favorite (testing) Podcast.
    • Two weeks ago I interviewed with Joe Colantonio for what will eventually become an episode of his TestTalks podcast. I’m really excited to see how this turns out!

That’s enough about me, what have you been up to?

 

Quality Equals Profit – Understanding Quality Helps Us Test

A coworker sent me an email about a conversation he had with one of our executives about risk and how quality played into her decision making:

“She [the executive] was giving me some insight…

As we were discussing the balance of quality policies (as we experienced them in the beta program) and balancing them against expediency she mentioned that quality equals profits. Ultimately this is because quality means a greater degree of predictability, and that greater certainty influences commercial policy to take risk out of competitive pricing, which means more deals won, and also means revenue won’t be eroded by fixing product issues once deployed (and the negative marketing spin they can create).”

How fascinating! This executive thinks quality is about predictability and reducing the risk of selling! I’ve worked at a few large companies where teams of programmers, analysts, testers, support people, professional services and others are brought together from various functional areas for a particular project. I’ve also worked at small startups where I sit next to all my engineers and in each case, regardless of the organization size the testing process mostly works the same way.

As things are scoped (through planning or building), the information that flows out from the project help us testers start to understand the product under test, how much we’ll have to test (scope), what our requirements are (both written and not), who makes up the team, et. al. But it’s been my experience with all these processes going on we end up forgetting or missing the important thing for understanding our project:

QualityEqualsProfit1

Our mission as understood by our stakeholders! For testers this mission will come in the form of an information objective (or multiple objectives). Our information objective is very important because it dictates the problem or problems we are trying to solve for our customers and/or stakeholders (who I’ll refer to now on collectively as stakeholders). To make matters more complex it changes with each project even if we’re working on the same product or with the same team.

To help understand our testing objectives and our overall mission we should be asking what do our customers expect from this project? Just because some written requirement states something is supposed to work one way, does that mean our stakeholders really want it? Would they in fact accept something else that addresses their problem in a slightly different way? Matt Heusser uses the term “desirements” instead of requirements because when you really understand your stakeholders you realize they are willing to make changes if they meet the end goal.

As we begin to understand our testing objectives we then need to understand from those same stakeholders what quality means to them. If you are like me, you’ve probably worked on many teams and/or projects where the assumption was “whatever I deem to be quality is quality”.  For example, in the quote at the beginning it sounds like this executive is interested in quality as it relates to controlling costs and reducing risks (at least the risk of spending more money to fix things later). This could lead to more questions like what information do they need (information objective) that would help them make better decisions about controlling costs and/or spending more money to fix problems later? Is it possible that testing can help uncover this information?

Perhaps by understanding what problems (or types of problems) exist in the system now we can make fixing problems in the future more predictable.

If this was the consensus across our stakeholders and/or if this stakeholder was the stakeholder that mattered this information objective would drive our mission and our test strategy. If this wasn’t the stakeholder who matter but was just one, we’d also want to know what the other stakeholders think was important. What is quality to them? What information do they need from testing?

QualityEqualsProfit2

Quality is subjective and we should expect to get a different answer from each of your stakeholders but that’s ok. Knowing what quality is, what each group of stakeholders expects and to whom certain things are important can lead to a better understanding of the priorities of the test efforts. At the very least it will influence how bugs are fixed.

Influence how bugs are fixed? Yes. Let me give you an example.

Say we have a stakeholder who thinks quality means the branding of the system (logos, trademarks, text with the company name) conforms to the latest design guidelines and we know this because we asked their group what quality meant and what information they needed to see to sign off on the release. If we (the testers) file a bug that says something like “logo isn’t the appropriate size as in design guidelines” its entirely possible a programmer would consider this low priority and skip it. If this stakeholder is important to the project team we can cite who would be affected and the impact of NOT fixing the bug. Again assuming the importance of the stakeholder this might be enough evidence to the programmer to rate this bug a higher priority.

The opposite could also be said. By knowing what our stakeholders expect from the testing mission we can also identify biases certain groups will have.

Going back to the example in the beginning if we know our executive thinks quality is profit and wants testing to provide information to help control costs, we know they may be unlikely to get behind fixes or features that directly benefit another stakeholder if they don’t fall into this category. It doesn’t matter how beneficial to that other stakeholder or group of stakeholders, so we may need to provide additional information in bugs or change requests to address that bias.

It comes down to: The more I know about my stakeholders the greater effect it will have on my mission and my testing objectives and eventually it will make it easier to appeal and influence the person making the decision about whether to fix a bug or implement a change.

Testing is often too large of a task to just randomly go through the testing process without thinking about or trying to understand the underlying mission and objectives of testing, instead it requires sampling. Asking what quality means to your stakeholders is one way to sample. Language influences the way we think about and deal with problems.

It may sound odd, as testers, to equate quality to profits (maybe you think quality is reputation, credibility, no UI bugs, no Easter eggs, etc.) but if we don’t ask these basic questions we may misunderstand our testing mission.

However if we do ask questions like who are the people that matter, what do they think quality is and what do they need to understand from testing, then we end up dealing with the right problems for the right people.

These days I make it a point to understand my stakeholders and talk directly to them, even the obvious ones I might be sitting next to because so much of what I do is effected by the information they give to me. I ask them what quality means and what information is important for them to understand which leads me to develop my information objectives, my mission and eventually my test strategy.

This article was originally published in the December 2013 edition of Testing Circus Magazine. It has been updated and reposted because it highlights an important concept I was grappling with in 2013 which, in 2019 is that customers are the only ones capable of judging and evaluating the quality of our product. Enjoy!

2018 In Review

As it has become a yearly tradition I will attempt to summarize the most popular and important articles I’ve written over the year along with some other forward-looking (and likely wrong) statements mixed in with past reflections.

You can find previous years here: 2017 | 2016 | 2015

What happened in 2018?

So much seems to have happend in 2018! I lost a job (laid off by a startup having problems – a startup on the brink of closing) but quickly found work as an Automation Engineering at BloomNation. That has led to new and interesting challenges both balancing time with my family and commitment to work. I’m learning lots of new things including how to program in JavaScript and the intracacies of SEO.

I’ve always spent a lot of time volunteering for the non-profit Association for Software Testing but seemed to take it to new highs as I recruited, promoted and led what I like to call “Season 1” of our webinar program. I also taught a class or two for AST-BBST and was elected to the Board of Directors!

Our home came very close to burning down in the most recent Southern California Wildfires (fires were stopped a few hundred yards away). Oh and I did a little writing:

The Five Most-Viewed Articles:

  • How To Run Your Selenium Tests Headlessly in Docker – A guide to setting up your test automation framework with a Chrome docker container for running your Selenium tests headlessly, complete with code examples. This has been my default configuration for the last year or so due to its simplicity.
  • Good and Bad UI Test Automation explained – Richard Bradshaw went off on a Twitter deep dive on UI test automation and the subtitles involved with doing it well. I enjoyed the tweetstorm and the topic so I decided to annotate those tweets and provide more context. I think I have enough information now to do a part 2 of this!
  • How to set up Apple Pay on Mac (non TouchID) – I was testing Stripe’s ApplePay integration but for some reason I wasn’t seeing ApplePay enabled through Safari. Googling didn’t help so after I figured it out I wrote a how to guide.
  • 8 Tools I use to Accelerate My Testing – The Modern Testing Principles introduced the concept of accelerating the achievement of shippable quality. Along those same lines I wrote about the tools I was using to accelerate my testing work.
  • A typical day of Testing – How I worked in 2018, aka what a typical day of testing looked like for me. It’s amazing what a year difference can make!

The first two articles made it into my top 10 articles over all, which is great. Traffic to this site continues moving up and to the right over time. In 2018 alone I had more than 100k page views. So crazy!

A few other Important Articles:

  • After attending TestBash in San Francisco in November my family and I came home to find the surrounding areas of our home on fire. I find writing helps keep me calm and allows me to vent a bit so I wrote about the experience.
  • I enjoyed TestBash so much I wrote a recap of both Day 1 and Day 2.
  • Most of the year I wrote for myself and for WonderProxy but I also wrote one article for Stickyminds on Participating in Code Reviews as a Tester. Turns out it was one of the top 10 articles they published in all of 2018!

Highlights:

  • Hosted 6 webinars for the AST on a variety of topics. Not bad for part time thing. I’ve got a lot of ideas for 2019 too!
  • Elected to the Board of Directors for the AST. I’m now the Treasurer!
  • Attended 2 in person conferences, TestBash SF and CAST2018.
  • Gave a workshop at CAST2018 on the test technique Domain Testing.
  • Flew to Des Moines, IA and gave a workshop on Git + GitHub to a local meetup there.
  • Published two articles per month on average for this site, plus another five elsewhere. As I mentioned above the Stickyminds article was the 6th most popular of 2018!
  • Started a new job as an Automation Engineer at BloomNation!

The Future (aka 2019):

Predicting the future is fun and yet meaningless. But here are the things already on my radar for 2019:

  • Publish at least 2 articles per month here. This pace was challenging for 2018 but doable so my goal is to continue this and maybe do some outside writing.
  • Still pushing my understanding of when and how to use data to better understand our customers. After all Customers are the only ones who can judge the quality of our software.
  • Finally start updating AST’s BBST program. Working with my AST partner in crime Simon Peter Schrijver we’ve got some big plans to update content.
  • Finally migrate this blog to DigitalOcean & begin monitizing with sponsorships!
  • Continue practicing JavaScript development with NodeJS

Cheers to the rest of 2019! What will you be doing?

Remember remember it’s the month of November

November has been so packed full of interesting events that it’s time for an update! (Although perhaps not as interesting as Guy Fawkes Night, on November 5th).

The first weekend of November I took my position as incoming Treasurer on the Board of Directors for the Association for Software Testing. We (the board) spent the weekend planning out the 2019 year at a high level including the next CAST conference and a number of updates across our education and marketing fronts. I’m happy / scared to say I took on a lot of new challenges. It’s interesting to get the high level view of the organization that I’ve spent so many years volunteering for.

The second week of November saw me hosting an AST webinar on API Testing with Jason Ioannides of API Fortress that exceeded all my expectations. We had a record number of sign ups and attendees. We couldn’t fit everyone in but we did record the talk which is now on the AST YouTube channel. We’ve been on a roll with our Webinars, personally I find them fun and I think we’ve been creating content people enjoy. Got any ideas for future webinars? (Leave me a comment or two)

Also last week I was in San Francisco attending the first California located TestBash conference. It’s my first TestBash and of all the different conferences I’ve been to it’s certainly unique. TestBash’s format is unique for conferences -> a single track over 2 days with each presentation being 30-45 minutes. On the plus side that means a lot of presentations and topics with the chance to impress. The downside is that there isn’t much time for asking questions or conferring unless you do it during the break times.