Remember remember it’s the month of November

November has been so packed full of interesting events that it’s time for an update! (Although perhaps not as interesting as Guy Fawkes Night, on November 5th).

The first weekend of November I took my position as incoming Treasurer on the Board of Directors for the Association for Software Testing. We (the board) spent the weekend planning out the 2019 year at a high level including the next CAST conference and a number of updates across our education and marketing fronts. I’m happy / scared to say I took on a lot of new challenges. It’s interesting to get the high level view of the organization that I’ve spent so many years volunteering for.

The second week of November saw me hosting an AST webinar on API Testing with Jason Ioannides of API Fortress that exceeded all my expectations. We had a record number of sign ups and attendees. We couldn’t fit everyone in but we did record the talk which is now on the AST YouTube channel. We’ve been on a roll with our Webinars, personally I find them fun and I think we’ve been creating content people enjoy. Got any ideas for future webinars? (Leave me a comment or two)

Also last week I was in San Francisco attending the first California located TestBash conference. It’s my first TestBash and of all the different conferences I’ve been to it’s certainly unique. TestBash’s format is unique for conferences -> a single track over 2 days with each presentation being 30-45 minutes. On the plus side that means a lot of presentations and topics with the chance to impress. The downside is that there isn’t much time for asking questions or conferring unless you do it during the break times.

BBST Domain Testing – An Experience Report

The Domain Testing Workbook

In late January of 2014, after the Workshop on Teaching Software Testing (WTST) at Florida Institute of Technology, Dr. Cem Kaner and Dr. Rebecca Fiedler put together a 5-day pilot course to beta test a new Black Box Software Testing (BBST) course called Domain Testing. I was one of ten participants to try it out.

Two series of BBST

  • The first BBST series (Foundations, Bug Advocacy and Test Design) came from research funded by the National Science Foundation and Dr. Kaner. Part of the agreement with the NSF was to make the materials open source while also creating a way to teach the materials online with high standards. The Association for Software Testing (AST) was the lab for the initial classes and they continue to teach them to this day.
  • BBST Domain Testing represents the next step in a second BBST series focusing on specific test design techniques (Domain, Scenario and Specification based testing). Unlike the first series, the NSF didn’t fund the development of these classes so the materials won’t be open sourced. They also won’t be taught through AST, instead they will come directly from Dr. Cem Kaner’s corporate training firm Kaner Fiedler & Associates (or KFA for short).

What is domain testing?

Although domain testing sounds like it relates to domain knowledge, it is an umbrella term for equivalence class analysis (partitioning) and boundary testing. Domain testing is a sampling strategy where possible values of a variable (anything that can change) are divided into a subset of values that are in some way equivalent (equivalence class analysis). Then tests are designed to only use one or two values from each subset, along with boundary and extreme values that increase the likelihood of exposing bugs.

Why use domain testing?

Designing tests is about making choices. Choices such as:

  • How many tests do we want to run (from an impossibly large set of potential tests)?
  • How powerful are the tests we have chosen?
  • What do we hope to learn from the tests we’ll run?

When used appropriately domain testing can help us increase our efficiency by helping us run less redundant tests and increase our effectiveness by helping us find bugs thanks to powerful tests.

The Workbook

The Domain Testing Workbook (available on Amazon or directly from Context Driven Press) was published for those interested in self-study but also with the intention of using it in a class. The book contains among other things, a schema (list of ideas or step by step sequence of questions to answer) and examples ranging from the easy, classic problems, to increasingly more difficult examples. During class I used the book frequently for background context, explanations of concepts and examples, details of the Schema and for the many ideas sitting in the books multiple test catalogs.

The Class

Prior to the class, my experience applying domain testing consisted of trying to overflow input and output fields. The problem with the way I was doing things, besides being ad hoc, was I didn’t utilize the power of domain testing as a sampling technique. That’s what inspired me to take the class.

The pilot class was in-person for five full days at Florida Institute of Technology’s campus. Dr. Kaner delivered lectures in the morning and afternoon followed by exercises after the lectures and concluded the day with homework for additional practice. For example, on our first day, we learned how to characterize variables and worked through classic examples that originally appeared in the book Testing Computer Software.

Our first assignment was to demonstrate we were comfortable doing a variable tour (variables the basis for domain testing) of our sample application, FM Starting Point (a FileMaker Pro template), using Xmind. After the tour it was time to classify a smaller set of variables (a dozen) based on their data types; determine their primary dimensions and what benefit, if any, would come from doing domain testing. Finally that information went into a classical table.

On our second day we pushed further. We continued to classify variables and build classical tables only with more complexity than before. Continuing with the Schema we turned to creating a risk equivalence table. Risk equivalence tables are essentially risk-based versions of the classical table but you explicitly talk about the risks (or failures) your tests hope to expose and therefore describe why your test designs are powerful. The challenging and time consuming aspects of creating risk or classical tables is coming up with failure modes for tests – luckily the workbook has catalogs you can use for inspiration.

Our third day continued our practice of variable analysis; creating classical and risk based equivalence tables but added in pair testing of independent variables using the ACTs tool (Microsoft’s PIC tool that we used in Test Design was also an option). Our final days in the class focused on putting together the individual things we learned from the Schema, culminating in a capstone project.

The capstone project allowed us to showcase the skill and understanding we gained from the class into action by choosing a piece of an application and working through each step of the eighteen step schema. It was a difficult assignment but certainly the most fun part of the class. I made a lot of mistakes as I went through the steps without the assignments as a guide but the capstone allowed me to figure out what I like and don’t like, what order to address aspects of the schema (I didn’t like going in logical order) and going forward I’ll be more efficient and effective.

Future Classes

The pilot was a bit different from how BBST Domain Testing courses will be handled in the future. Besides the obvious in-person versus online transition, there will likely be a wide range of example applications to practice domain testing on.

At the time of the pilot (January of 2014) the class had two finished sets of real world example videos – QuickBooks (account software) and Electric Quilt 7 (design software). There were plans to add more real world examples including a Sewing machine (not your grandmothers sewing machine but a modern one to serve as an example of an embedded device), a video game (for a glassbox approach), an investing application and a database application.

Taking the class changed the way I look at testing

I barely remembered the introduction of domain testing in BBST Test Design after the class was over – I just didn’t get it. Things went by so fast, at such a high level that made it difficult to understand how to address things properly. After the class was over I couldn’t apply it to my work. This time things are different. After BBST Domain Testing, using the Schema and workbook I get domain testing.

When I look at an application today, I have a strong sense of where the strategy can be applied and where it shouldn’t. That makes me more confident in my abilities (not over confident, I’m still a novice), gives me lots of new and interesting ideas and a place to start practicing. More importantly I can actually see myself applying the technique to the software I test on a regular basis.

When Becky and Cem asked what I thought of the class I said:

it wasn’t as easy as I thought, but it was fun!

References

  1. Kaner, Padmanabhan & Hoffman. The Domain Testing Workbook. Context-Driven Press, 2013.
  2. Kaner, Cem. Teaching Domain Testing: A Status Report.

This article was originally published in the February 2014 edition of Testing Circus Magazine. This was such a great learning opportunity I wanted to highlight it by slightly updating and reposting it.

How To Run Your Selenium Tests with Headless Chrome

The Problem

If you want to run your tests headlessly on a Continuous Integration (CI) server you’ll quickly realize that you can’t with an out-of-the-box setup since there is no display output for the browser to launch in. You could use a third party library like Xvfb (tip 38) or PhantomJS (tip 46) but those can be hard to install and aren’t guaranteed to be supported in the long run (like PhantomJS).

A Solution

Enter Headless Chrome. Headless is simply a mode you can put Chrome into that allows it to run without displaying on screen but still gets you the same great results. This is a better option than using Chrome in a Headless manner such as in a docker container where the the container actually uses Xvfb.

Starting with Chrome 59 (Chrome 60 for Windows) we can simply pass Chrome a few configuration options to enable headless mode.

An Example in Ruby

Before we start make sure you’ve at least got the latest version of Chrome installed along with the latest version of ChromeDriver.

Let’s create a simple Selenium script (the example is posted below).

  1. We will pull in the requisite libraries and then create our setup method where we will pass Chrome our headless option as a command line argument. The first add_argument of ‘– headless’ allows us to run Chrome in headless mode. The second argument is, according to Google, temporarily required to work around a few known bugs. The third argument is optional but gives us the ability to debug our application in another browser if we need to (using localhost:9222).
  2. Now let’s finish our test by creating our teardown and run methods:

Here we are loading a page, asserting on the title (to make sure we are in the right place), and taking a screenshot to make sure our headless setup is working correctly. Here’s what our screenshot looks like:

headless

Expected Behavior

When we save our file and run it (e.g. ruby headless_chrome.rb) here is what will happen:

  1. An empty chrome browser instance will open
  2. Test runs and captures a screenshot
  3. Browser closes

Outro

Hopefully this tip has helped you get your tests running smoothly locally or on your CI Server. Happy Testing!

Additional References

This was originally written for and posted on the Elemental Selenium newsletter. I liked it so much I decided to cross post it with some updates.

Oh and if this article worked for you please consider sharing it or buying me coffee!

Appropriate Test Documentation & Formatting

The Question

Recently in an online forum a tester person asked:

Does someone have a simple example of test case (excel sheet) format? I am the only one tester in my company and we are trying to arrange the test documentation. Any advice or example will be useful.

I wish someone had given me this advice when I first started out:

Be careful about using someone else’s formatting and/or templates. If you don’t know what you are doing, you might be inheriting their issues and misunderstandings. It’s usually best to understand what you want to accomplish and then find a solution for yourself. Especially if it’s something simple like using Excel for documentation. Tables and matrices are great for organizing tests and don’t cost much money or time to make.

Appropriate Test Documentation

The appropriate documentation for your tests (or test cases) will depend to a large degree on the test technique and approach you use. Test Case Management systems (and similar systems) are often designed for functional or scenarios tests where the format is similar and you can list out multiple steps. Other test techniques such as Path Analysis, Domain Testing and even Risk Based testing can look very different when they are well designed. The same can be said when you decide to take an Exploratory approach to your testing where you are likely to write Exploratory Charters. Cramming all of these different techniques into one system usually isn’t the best idea.

The Smallest Amount

Focus on the smallest amount of documentation you need to do your job well. You have a finite amount of time and if you spend too much of it creating documentation you will have less time to do the work. If you are writing tests for other humans to read and understand, they should be written in a way that communicates this well such as using visuals (mind maps, diagrams, etc.) and summaries. Similarly if you are writing the tests for computers to read and understand they can (need to be) much more granular and step by step.

I’d wager most experienced engineers have gone through this process of: trying to document as much as possible but by the time you are done you didn’t do enough testing and the documentation isn’t done either. Maybe you put it all into a test case management system that no one else cares to read because it’s in this weird format that probably only makes sense to you.

Staying Agile

Focusing on the appropriate documentation for your tests and then creating the smallest amount you need to do your job well also allows you to have agility. Agile is all about being able to adjust to changes as they come in. Quick changes in direction aren’t kind on large amounts of pre-scripted documentation and/or can result in lost work.

I wish this approach was more obvious but a quick google search will reveal it’s not the dominate theory. However, if you read this and take the advice you’ll be learning from my mistakes. Focus on using the smallest amount of documentation you need to be successful and making sure that documentation is appropriate for your test choices will give you agility and future proofing even as you scale your team size and move into the future.

How To Run Your Selenium Tests Headlessly in Docker

The Old Way

It used to be that in order to get your Selenium tests running on a given machine you had to install each individual browser and then the browser drivers (for instance ChromeDriver for Chrome). Some of my most popular posts are about installing these drivers. However that’s all changed if we instead use containers. Simply download and run a single Docker container with both the the browser and the driver pre-installed.

Problem solved!

…Except now we’ll be running your tests headlessly which customers don’t do.

The Catch: Headless Browsers

In the context running a Selenium test headlessly in Chrome, it means Chrome launches and runs the test in the browser but you don’t see it because there is no GUI. Instead you get a command line to run and debug your tests. The downside is one customers don’t use your application in a headless state and two these tests are harder to debug.

The upside is we can manage the installation and running of the browsers and drivers in an easier way. We can also run our tests on devices that don’t have computer graphics like AWS EC2 instances (virtual machines), and CI/CD services like TravisCI and CircleCI to name a few.

Docker Desktop and Container Installation:

In this exercise we will download a single (standalone) Chrome browser with ChromeDriver already configured. Then we will simply run a test!

  • First thing we need to do is install Docker Desktop for our respective OS (Mac or Windows)
    • Once this is up and running we can download our container
    • (Note: this is a desktop app but can also be installed from the command line for future virtual machine use)
  • Second, since we already know what container we want, we simply need to copy and paste this command into our terminal docker pull selenium/standalone-chrome
    • This will download the Chrome container image
  • Third, we run is to start the container so Chrome is running
    • We can run docker ps at any time to see if any containers are running
    • We now run
      docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome
    • This sets the docker container to use the port 4444. We will use this later in our test file.

Running Our Tests: An Example

Now we’re ready to write a test using Selenium and our now running Docker container with Chrome in it. For fun I’m going to write this in Ruby but it should work for any language:

Expected Behavior

When we save our file and run it (e.g. ruby headless_chrome.rb) here is what will happen:

  • A chrome browser session will open within the docker container
  • Test runs and saves a screenshot
  • Browser closes
  • All we see is the result of the test

There you go! We managed to install a container with both Chrome and ChromeDriver installed on it and then wrote and ran a test to demonstrate it works. You’ve now run a Selenium Test Headlessly with Docker!

In the larger picture this means each time we need to install Selenium on a new computer (CI, EC2 instance, virtual machine, desktop) it’s simply about installing docker and downloading an image, spinning it up and connecting it to your test. After having done this half a dozen times I think it has huge advantages over the old ways (link to Installing ChromeDriver on macOS).

References:

  • Selenium Docker Hub for finding the approprite container
  • This is not the same as using Headless Chrome (which is a configuration option in modern versions of Chrome). To learn more about using Headless Chrome go here.

 

Oh and if this article worked for you please consider sharing it or buying me coffee!

Highlights from CAST 2018

Last week I attended CAST in Cocoa Beach, Florida, which was my second time attending and the first since CAST in Grand Rapids back in 2015. It was a fun experience for a number of reasons including giving my first workshop at CAST and being elected to the AST Board of Directors!!! Here are some highlights:

  • Dwayne Green and I hosted a workshop called A Quick Introduction to Domain Testing about applying the test technique to a few sample applications. The workshop went well given the limited amount of time we had and for trying to teach a complex topic with a hands on approach. We’re working on a newer version that is a half day tutorial for next year which we believe will be much better. The upside is we got roughly 35 people to do some hands on testing and present their findings after each session!
  • Despite the tiring nature of the travel (I was only at the conference for two days), I walked away feeling energized about having met new people, came face to face with people I know purely online (and now AFK) and took a few things away from the talks.
  • Like many others in attendance I heard of Jerry Weinberg’s passing. While I never had the pleasure to meet him in person, I have read a few of his many books and am aware of his huge influence on our industry and community.
  • Gave an even more brief lightning talk on the Modern Testing Principles. Was pleasantly surprised when I asked the packed crowd how many had heard of the principles around roughly 20% or so of the audience had! (Afterwards I had a few follow up conversations about the principles as well.)
  • I was lucky enough to be elected to the Association for Software Testing’s Board of Directors along with a few fantastic people I already collaborate with. Thank you for everyone who voted! Our new terms starts in October and I will be taking on the role of Treasurer! I also got to sit in on my first board meeting as a director elect (I didn’t participate since it’s not my term yet).

 

I’m running for the 2018-2020 AST Board of Directors

I’m running for the 2018-2020 AST Board of Directors this year. I’m asking for your Vote! (If you are an AST member you’ll see an email for voting on August 6th!)

If my name sounds familiar to you it could be because I ran and failed to be elected to the AST Board of Directors last year. (If at first you don’t succeed…?) Or it might be because I run and host AST’s webinar series which you might have attended, seen a blog post about or even read a newsletter article about. Or maybe I answered a question you sent to the AST via a web form, or maybe I taught your BBST class.

The point is I’ve been around helping out AST for a long time because I in it’s mission, only I think I can do more and I’m willing to dedicate my time to do so.

As part of the nomination process all of the candidates answered member-submitted questions. Please read mine and let me know what you think (even if you aren’t an AST member.)

After the questions were published a few things came to mind and might influence whether or not you’d vote for me:

  • This current slate of candidates isn’t as diverse as it could be because no women are running. From the sounds of it, the women and PoC who the board reached out to didn’t want to run or weren’t qualified. This makes the diversity question even more important and goes back to my original point -> we need to have a diverse membership in order to be able to tap those people for board candidates.
    • I love the questions regarding diversity. It’s a hard problem but I like that we are actively thinking about it.
  • CAST, the Conference for the Association for Software Testing, is partially PayToSpeak (the amount they cover isn’t enough for international speakers) which can have an impact on the diversity of speakers who can attend (only those who can afford to travel on their own dime). If elected I’m definitely going to change this.
  • When I ran (and lost) last time, one of the board members who won didn’t continue into the second part of their term. This happened the year before as well. Basically what I’m saying is this is a tough job but I’m willing to do it.

All of this is a way to remind and ask those AST members to vote for me when the time comes!

 

8 Tools I use to Accelerate My Testing

Inspired by Justin Rorhman’s post of a similar name with a slight twist focusing on tools that generally help accelerate my testing.

As a test and quality specialist embedded in an engineering team I have a lot of work to do on any given day. Our engineering team’s goal is to ship quality software on a regular basis to deliver value to our customers. Naturally I rely heavily on a number of tools to help me better understand what is going on with the software or code under test and/or to accelerate what I’m able to do in a given amount of time.

Like Justin said in his article “Testing tools don’t make me a good tester, but they do make me a better tester. These tools help amplify things I need to do repeatedly or give me access to the product I wouldn’t have otherwise. They also shape the way I think about testing problems, for better and worse.”

In no particular order these tools are:

  1. GitHub. We use the GitHub workflow to deliver all of our code including application, library, and test automation. With each push to a particular branch our CI system builds the application and runs our unit and integration tests. With each Pull Request I can see all or just specific changes, code reviews and test run summaries. All this helps keep our build pipeline clean, makes it easier to identify potential issues quickly and helps me quickly build my test strategy.
  2. Docker. Each separate application of our system has been dockerized (aka runs in a container). This makes it incredibly easy to set new people up (a few hours of setup instead of a day) and gives us some powerful logging for each application. It also reduces conflicts when switching between branches where libraries or decencies have been changed.
  3. Heroku review apps. As I mentioned above with each push our application is compiled through our CI system. Once finished our CI system pushes that complied build to Heroku which automatically provisions a staging like environment accessible through its own url. (This feature is called a Heroku Review application). In about 5 minutes after a set of changes have been built our team now has a brand-new test environment ready for those hard to test local problems like testing on real mobile devices or a few special integrated services. Being able to build these new automatic quickly allows our developers to show off / get quicker input to the work they are doing.
  4. Browser DeveloperTools. I primarily use Chrome and Safari but the developer tools in each browser are priceless. From Inspecting requests and looking at their data, to setting or removing experiment variables, debugging problems, looking for problems, profiling, responsive design, layout, checking DOM elements, etc browser DeveloperTools really are the first place to start. It would be so much harder to learn what is going on without them.
  5. CircleCI. Great for keeping your deployment pipeline clean, running your build tests and ensuring those changes get where they are going each time. It doesn’t much matter what CI system you use it’s helping to accelerate something that would normally take away from developer or tester time.
  6. EverNote. Surprising or not, I’m always taking notes about any number of things like quick terminal commands, operations for creating data, migrating databases in heroku, setting up configurations, test ideas, questions for follow up, etc. Anything I might forget goes here and I can get to it anywhere, any place on any device.
  7. iTerm2. A command line replacement utility with Oh My Zsh installed that I’ve customized and continue to customize with shortcuts and aliases to make my repetitive tasks quicker. Also customized with visuals showing which repo I’m in, what branch I’m on and if there are uncommitted files, because I do enough on the command line.
  8. Automated UI acceptance tests. Running our acceptance tests for each staging deploy (or set of deploys) allows us time to do more deep exploration on the application as a whole with a reduced focus on those areas covered by the existing tests.

Part of the desire for writing this was to record what my work looks like now and then be able to compare it later-on in the future. However, in the time since I started writing this article I changed jobs and now the future is here. Well, kind of. My new team’s development stack is a lot more varied, which means so are the tools I’m using. I think it’s still best to wait for half a year or so before I write a follow up post on the tools accelerate that work. In the meantime, what tools do you use that accelerate your testing?

Good and Bad UI Test Automation explained – Inspired by Richard Bradshaw’s Tweets

Generally speaking there’s a scary trend with the influx of people interested in test automation where (seemingly) everyone wants to automate at the UI level. For example the phrase “Test automation” seems to be synonymous with UI automation which seems to mean using Selenium. To be fair there are numerous reasons for this. First, if you primarily focus on functional testing through the UI, this application of automation is the most relevant and straightforward to understand. Second, there are a lot of resources out there (books, blogs, classes) for Selenium. Third it is much easier to write some blackbox automation at the UI level then to develop competence at writing automation at the API or unit level. (You should still learn to do this lower.)

So Richard took to Twitter to explain his thoughts. Let’s take a look:

Before we go very far, let’s look at what Richard means by targeted tests.

Tests are “targeted” if they are designed to reveal a specific risk or failure (we might call this risk-based). If you are doing risk-based testing, your tests should ALWAYS be designed this way, regardless of how you intend to execute the tests (automated or not).

Seams are how you use the system’s implementation to help you build more streamlined and realiable automated tests. Seams can be can be APIs, mocks, stubs, or faked dependecies of your real system.

Want some examples of Seams? Checkout this blog post by Angie Jones.

Yes, most times we can use better designed tests! Want to test the various ways to get to a particular feature? Use PATH testing. Want to test the feature itself? Use functional or domain testing, etc. Some test techniques will be better at surfacing one type of problem and bad at surfacing another. It’s not always necessary to design your UI tests as long feature rich Scenarios.

When we talk of interfaces and automating at the lowest level it is usually a good time to mention this is what the Test Automation Pyramid tries to describe. There are lots of versions of this but I recommend Alan Page’s Test Automation Snowman.

If you can find the failure by writing a test lower in the stack (at the API level intead of UI level or at the unit level instead at the API level), the faster it runs and the faster you get feedback. If you do have to write the test at a higher level like the UI can you find some way to decrease the setup time / data of the test? (See above for Angie Jone’s blog post.) If you aren’t sure maybe thinking about whether you are testing the UI or testing through the UI will help?

Yes! Adding testability like IDs, classes, data-test attributes, setting up test data, mocking, etc. should be done as quickly and dependbly as possible. If you can’t do this on your own (or you have a different process for making changes) bring it up during sprint planning in order to get the development teams help behind building a solution.

This IS a thing. Selenium is great, it’s open source and the WebDriver protocol is the soon-to-be de facto standard for interacting with browsers. However there are lots of tools, some will use Selenium under the hood and others that are completely different (think Cypress.io). Depending on the problem you need to solve, you may find a specific tool much better than Selenium. GitHub is a really good resource for finding open source tools and when you find something that might work, give it a try.

If you want to write Good UI Test Automation your tests should be ideally suited for the UI (and not some other layer), targeted, reliable, use seams where possible andcover a specific risk. If you aren’t here yet work to refactor your test code until you can get there.

Did I miss anything about Richard’s tweet storm that was important?

Opting out of A/B Tests while Running your Automated Tests

At Laurel & Wolf, we extensively introduce & test new features as part of A/B or split testing. Basically we can split traffic coming to a particular page into different groups and then show those groups different variations of a page or object to see if those changes lead to different behavior. Most commonly this is used to test if small (or large) changes can drive some statistically significant positive change in conversion of package or eCommerce sales. The challenge when running automated UI tests and A/B tests together is that your tests are no longer deterministic (having one dependable state).

Different sites conduct A/B tests differently depending on their systems. With our system we’re able to bucket users into groups once the land on the page which then either sets the AB Test and its variant options in local storage or as a cookie. Here are a few ways we successfully dealt with the non-determinism of A/B Tests.

Opting out of A/B Tests (by taking the control variant):

  1. Opt into the Control variant by appending query params to the route (e.g. www.kenst.com?nameOfExperiment=NameOfControlVariant). Most frameworks offer the ability to check for a query param to see if it should force a variant for an experiment and ours was no exception. For the tests we knew were impacted by A/B tests we could simply append to the base_url the query params for the experiment name and the control variant to avoid changing anything. This method didn’t require us to really care about where the experiment + variant were set (either as a cookie or localStorage) but really only worked for a single A/B test for a given test.
  2. Opt into the Control variant by setting localStorage. We often accomplished this by simply running some JS on the page that would set our localStorage (e.g.  @driver.execute_script("localStorage.setItem('NameOfExperiment', 'NameofControlVariant')")). Depending on where the A/B test sat within a test and/or if there were more than one, this was often the easiest way to opt out assuming the A/B test was set it localStorage.
  3. Opt into the Control variant by setting a cookie. Like the example for localStorage we often accomplished setting the cookie in the same way using JS (e.g.@driver.manage.add_cookie(name: 'NameOfExperiment', value: 'NameofControlVariant'). Again this was another simple way of allowing us to opt out of an experiment or even multiple experiments when necessary.

I know there are a few other ways to approach this. Alister Scott mentions a few ways here and Dave Haeffner mentions how to opt out of an Optimizely A/B test by setting a cookie here. How do you deal with your A/B tests? Do you opt-out and/or go for the control route as well?

Oh and if this article worked for you please consider sharing it or buying me coffee!