How to export environments from Postman

Postman can export data, including collections and environments, to be used outside of Postman. This is especially important when using Newman (their command-line collection runner).

With the release of Postman v8.2 it is easier to find and export Postman collections but harder to find and export environments, hence this article.

(more…)

Upgrading to WebDriverIO 6

In March I went through the process of upgrading to WebDriver v5. Last month I took the next step by upgrading our deployment to version 6 so we’d be current. I learned quite a bit from that first upgrade which made this upgrade a whole lot easier.

Here’s what I did to upgrade to WebDriverIO 6:

  1. Checked the Docs. I scanned through the change log sections on breaking changes. Maybe I’m boring but nothing in the docs breaking changes notes looked to impact our setup! 👏
  2. npm oudated . This is one of my favorite (and mostly unused) npm commands. It told me specifically what packages I had that were outdated. Hint: it was all of them. 🤯
  3. Upgrade the easy stuff.
    1. Based on what npm returned, I began by updating my package.json file for 3rd party libraries. Libraries such as prettier, chromedriver, moment, etc. that all played a part in my solution but didn’t deal directly with my tests. Then I npm install the latest changes.
  4. Run the whole test suite looking for failures.
  5. Upgrade the harder stuff.
    1. Same thing as step 3, updated the package.json versions to be the “latest” and then npm install.
    2. Another way to do this is to remove the node_modules folder and re-install each package based on the install instructions.
  6. Run the whole test suite… until nothing is broken!

More reasonable!

This upgrade wasn’t as daunting as I initially feared. With upgrades there’s always some level of concern you are going to “ruin a good thing”. Once I moved past that concern, there wasn’t much effort involved in getting things to work. Lots of credit goes to the WebDriverIO team for making the process more reasonable and straightforward, which of course they stated in their announcement:

This major update is much more reasonable and contains subtle changes that will help the project further grow while remaining performant at the same time.

I’ve been pretty happy thus far using WebDriverIO and I’m excited to see where things go!

How to take a screenshot and record video in macOS

One of the nice things about macOS are the built-in tools. A recently improved upon built-in tool is called Screenshot which allows users to take a screenshot and record video. Taking screenshots has been around macOS forever, but taking video directly from OS shortcuts came along in macOS Mojave. The challenge is remembering the commands.

This tutorial is an attempt to capture the quick commands / shortcuts to make this process easy.

(more…)

Upgrading to WebDriverIO 5

A few weeks ago I finished upgrading our implementation of WebDriverIO from version 4 to version 5. The impetus for the upgrade was an announcement by the WebDriverIO twitter account of a new beta version 6 to be quickly followed by a finished version (it’s already here). One thing was clear: you have to be on v5 to go to v6 and each new subsequent version would only be supported for a year. Time to upgrade!

I’d been on version 4 since I originally deployed WebDriverIO in mid 2018. I knew version 5 was out but I had no immediate plans to upgrade given all of the warnings around breaking changes.

Preparation

This isn’t to say I wasn’t preparing myself. I created a JIRA ticket to outline what the work might look like. I was going through the TestAutomationU course on WebDriverIO which uses V5 and of course practicing in my own repo. With confirmation of the EOL of version 4 it was time to move on. 

I scoped out more of the work in JIRA (and also made a story to upgrade to v6). I bookmarked a few important articles including the WebDriverIO blog post announcing version 5 which highlights, among other things, specifically How to Upgrade. Then I made note of the methods that were either changing or being replaced for easier reference.

Upgrading to WebDriverIO 5

Finally it was time for the upgrade itself. The first question I needed to answer was logistical: how do I approach making the changes? Create a whole new repo, install the new packages and then move my tests over? Or just upgrade in place? Despite the daunting feeling I had, I figured it would be easier to upgrade in place and deal with the test failing as they came. This would introduce less changes than actually trying to move things over. Then it was time to follow the recommendations in the blog post:

  1. Remove node_module
  2. Remove all wdio-* packages from package.json
  3. Install the latest version of webdriverio: $ npm install [email protected]
    1. Note: if you did this today, you’d want to npm install [email protected] however I’d recommend going straight to v6 instead of going to v5 and then v6.
  4. Install the new wdio testrunner: $ npm install @wdio/cli --save-dev
  5. I have multiple configuration files and so I didn’t need to back them up. Rather I created a new one as part of the webdriverio configuration wizard. Then I eventually migrated / pruned the original ones until they were what I wanted.
  6. Rerun the configuration wizard: $ npx wdio config

All the Broken Things

Bam! WebDriverIO 5 deployed. Kind of. The easy part was done, next up was running each test one at a time, starting with the easiest / shortest tests. (I usually start with low hanging fruit so that I build momentum). When a test failed, I’d find out where and why (due to a rename or deprecation) and make a change. Rise and repeat for the whole test suite.

If this sounds simple, that’s because it was. All it took was time to remember where things were and why something was done a certain way and then make a change. 90% of the time this worked fine.

Other times it was more complicated. In at least one instance I ran into a breaking change, the functionality that was there before didn’t exist in the new version. I commented out 2 tests and moved on. (Speaking of which, I need to open a bug report about this).

Benefits

While it’s never fun to take something that’s working and break it just to upgrade, it does have some side benefits. Such as…. I fixed a few remaining bad ternary statements. Then cleaned up some unused libraries / plugins that I installed for some reason but can’t possibly remember anymore. Also refactored a bit of my code to make things simpler. When I was going through the TestAutomationU course I got to see how the author structured her tests. Now I have a new task to break apart my larger tests into smaller more defined tests with less assertions. 

All of this is to say, once you start making changes to improve one thing, it can snowball and lead to even more improvements. 

If you liked this article consider sharing it or buying me a coffee:

How to debug problems on Mobile Safari

When you are developing and/or testing a web app on iOS, having the ability to inspect and understand what is happening in the browser is incredibly useful. For just this reason Mobile Safari can use the same developer tools that are built into Safari on macOS.

To start debugging you’ll need your iOS device, a mac and a lightning cable. Then start by making sure both devices are configured to allow this.

Configure devices:

  1. Launch Safari on your mac; open preferences and within the advanced tab click the checkbox for “Show Develop menu in menu bar”.
    1. Confirm, you should now see a menu called Develop in the top bar
  2. On your iOS device (iPhone or iPad) go to Settings > Safari > Advanced and slide the toggle on for Web Inspector.

Debug a site in Mobile Safari:

  1. Connect your iOS device (iPhone or iPad) to your mac
  2. On the device browse to your site in Mobile Safari
  3. On your mac’s Safari, under the Develop menu you should see an option for your iOS device (either called iPhone or iPad).
  4. In that sub-menu you should see the url of the website you want to debug. Click that url and it will open a new Safari window with the full Safari Dev Tools.
  5. Start your investigation!

Safari Dev Tools

A few weeks back I was testing changes to review submissions (buy a product, leave a review) and on Chrome desktop everything worked fine (including Chrome’s mobile emulator). However after testing on a real device, Safari didn’t behave the same way. Luckily we were able to debug using this same method and fix those remaining issues before any customers saw problems.

It just goes to show real device testing matters!

Oh and if this article worked for you please consider sharing it or buying me coffee!

Installing ChromeDriver on Windows

Installing on Windows 7 & 10:

The following are the best ways to install ChromeDriver on Windows so you can run Selenium:

  • The easiest way to install ChromeDriver is to use a package manager such as Chocolatey. You should really have a package manager if you don’t now:
    1. Install Chocolatey. I used the cmd.exe method and it worked really quickly.
    2. Install ChromeDriver choco install chromedriver
    3. Confirm it was installed by typing chromedriver —version into your command line and seeing that it returns a version number. If it errors, ChromeDriver wasn’t installed.
  • Or run Chrome & ChromeDriver in a container such as Docker. Simply download the combined container, start it and point your code at the right address. Here’s an example.

Which methods have you found the easiest or most success with? Which methods didn’t work for you? Please leave a comment below. 


If this article worked for you please consider following me on twitter, sharing it or buying me coffee!

How To Run Your Selenium Tests with Headless Chrome

The Problem

If you want to run your tests headlessly on a Continuous Integration (CI) server you’ll quickly realize that you can’t with an out-of-the-box setup since there is no display output for the browser to launch in. You could use a third party library like Xvfb (tip 38) or PhantomJS (tip 46) but those can be hard to install and aren’t guaranteed to be supported in the long run (like PhantomJS).

A Solution

Enter Headless Chrome. Headless is simply a mode you can put Chrome into that allows it to run without displaying on screen but still gets you the same great results. This is a better option than using Chrome in a Headless manner such as in a docker container where the the container actually uses Xvfb.

Starting with Chrome 59 (Chrome 60 for Windows) we can simply pass Chrome a few configuration options to enable headless mode.

An Example in Ruby

Before we start make sure you’ve at least got the latest version of Chrome installed along with the latest version of ChromeDriver.

Let’s create a simple Selenium script (the example is posted below).

  1. We will pull in the requisite libraries and then create our setup method where we will pass Chrome our headless option as a command line argument. The first add_argument of ‘– headless’ allows us to run Chrome in headless mode. The second argument is, according to Google, temporarily required to work around a few known bugs. The third argument is optional but gives us the ability to debug our application in another browser if we need to (using localhost:9222).
  2. Now let’s finish our test by creating our teardown and run methods:

Here we are loading a page, asserting on the title (to make sure we are in the right place), and taking a screenshot to make sure our headless setup is working correctly. Here’s what our screenshot looks like:

headless

Expected Behavior

When we save our file and run it (e.g. ruby headless_chrome.rb) here is what will happen:

  1. An empty chrome browser instance will open
  2. Test runs and captures a screenshot
  3. Browser closes

Outro

Hopefully this tip has helped you get your tests running smoothly locally or on your CI Server. Happy Testing!

Additional References

This was originally written for and posted on the Elemental Selenium newsletter. I liked it so much I decided to cross post it with some updates.

Oh and if this article worked for you please consider sharing it or buying me coffee!

How To Run Your Selenium Tests Headlessly in Docker

The Old Way

It used to be that in order to get your Selenium tests running on a given machine you had to install each individual browser and then the browser drivers (for instance ChromeDriver for Chrome). Some of my most popular posts are about installing these drivers. However that’s all changed if we instead use containers. Simply download and run a single Docker container with both the the browser and the driver pre-installed.

Problem solved!

…Except now we’ll be running your tests headlessly which customers don’t do.

The Catch: Headless Browsers

In the context running a Selenium test headlessly in Chrome, it means Chrome launches and runs the test in the browser but you don’t see it because there is no GUI. Instead you get a command line to run and debug your tests. The downside is one customers don’t use your application in a headless state and two these tests are harder to debug.

The upside is we can manage the installation and running of the browsers and drivers in an easier way. We can also run our tests on devices that don’t have computer graphics like AWS EC2 instances (virtual machines), and CI/CD services like TravisCI and CircleCI to name a few.

Docker Desktop and Container Installation:

In this exercise we will download a single (standalone) Chrome browser with ChromeDriver already configured. Then we will simply run a test!

  • First thing we need to do is install Docker Desktop for our respective OS (Mac or Windows)
    • Once this is up and running we can download our container
    • (Note: this is a desktop app but can also be installed from the command line for future virtual machine use)
  • Second, since we already know what container we want, we simply need to copy and paste this command into our terminal docker pull selenium/standalone-chrome
    • This will download the Chrome container image
  • Third, we run is to start the container so Chrome is running
    • We can run docker ps at any time to see if any containers are running
    • We now run
      docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome
    • This sets the docker container to use the port 4444. We will use this later in our test file.

Running Our Tests: An Example

Now we’re ready to write a test using Selenium and our now running Docker container with Chrome in it. For fun I’m going to write this in Ruby but it should work for any language:

Expected Behavior

When we save our file and run it (e.g. ruby headless_chrome.rb) here is what will happen:

  • A chrome browser session will open within the docker container
  • Test runs and saves a screenshot
  • Browser closes
  • All we see is the result of the test

There you go! We managed to install a container with both Chrome and ChromeDriver installed on it and then wrote and ran a test to demonstrate it works. You’ve now run a Selenium Test Headlessly with Docker!

In the larger picture this means each time we need to install Selenium on a new computer (CI, EC2 instance, virtual machine, desktop) it’s simply about installing docker and downloading an image, spinning it up and connecting it to your test. After having done this half a dozen times I think it has huge advantages over the old ways (link to Installing ChromeDriver on macOS).

References:

  • Selenium Docker Hub for finding the approprite container
  • This is not the same as using Headless Chrome (which is a configuration option in modern versions of Chrome). To learn more about using Headless Chrome go here.

 

Oh and if this article worked for you please consider sharing it or buying me coffee!

Opting out of A/B Tests while Running your Automated Tests

At Laurel & Wolf, we extensively introduce & test new features as part of A/B or split testing. Basically we can split traffic coming to a particular page into different groups and then show those groups different variations of a page or object to see if those changes lead to different behavior. Most commonly this is used to test if small (or large) changes can drive some statistically significant positive change in conversion of package or eCommerce sales. The challenge when running automated UI tests and A/B tests together is that your tests are no longer deterministic (having one dependable state).

Different sites conduct A/B tests differently depending on their systems. With our system we’re able to bucket users into groups once the land on the page which then either sets the AB Test and its variant options in local storage or as a cookie. Here are a few ways we successfully dealt with the non-determinism of A/B Tests.

Opting out of A/B Tests (by taking the control variant):

  1. Opt into the Control variant by appending query params to the route (e.g. www.kenst.com?nameOfExperiment=NameOfControlVariant). Most frameworks offer the ability to check for a query param to see if it should force a variant for an experiment and ours was no exception. For the tests we knew were impacted by A/B tests we could simply append to the base_url the query params for the experiment name and the control variant to avoid changing anything. This method didn’t require us to really care about where the experiment + variant were set (either as a cookie or localStorage) but really only worked for a single A/B test for a given test.
  2. Opt into the Control variant by setting localStorage. We often accomplished this by simply running some JS on the page that would set our localStorage (e.g.  @driver.execute_script("localStorage.setItem('NameOfExperiment', 'NameofControlVariant')")). Depending on where the A/B test sat within a test and/or if there were more than one, this was often the easiest way to opt out assuming the A/B test was set it localStorage.
  3. Opt into the Control variant by setting a cookie. Like the example for localStorage we often accomplished setting the cookie in the same way using JS (e.g.@driver.manage.add_cookie(name: 'NameOfExperiment', value: 'NameofControlVariant'). Again this was another simple way of allowing us to opt out of an experiment or even multiple experiments when necessary.

I know there are a few other ways to approach this. Alister Scott mentions a few ways here and Dave Haeffner mentions how to opt out of an Optimizely A/B test by setting a cookie here. How do you deal with your A/B tests? Do you opt-out and/or go for the control route as well?

Oh and if this article worked for you please consider sharing it or buying me coffee!

How to set up Apple Pay on Mac (non TouchID)

According to Bailey, Apple Pay availability was limited to about 3 percent of stores in the U.S. when it launched in 2014, but is now accepted in 50 percent of stores. – Courtesy of AppleInsider.

At Laurel & Wolf we use Stripe to handle monies we collect from our customers. It’s a great service that takes care of many things including allowing us to accept Apple Pay. Although accepting Apple Pay means we need to test it, which I’ve been doing mostly from my iPhone. However I’ve been wanting to test it on my laptop and that means enabling it on macOS for Safari.

These instructions are for non TouchID macs. If you have a Mac with TouchID you can go to Apple’s website for easy set up.

Apple Pay Logo

Steps to enabling Apple Pay on non TouchID Macs

  1. On your iPhone confirm Apple Pay is active. If not set it up here.
  2. On your Mac sign-in to iCloud using your Apple ID. Without this you won’t have access to your Wallet’s cards.
  3. Allow websites to check if Apple Pay is set up on Safari.
    1. In Safari go to the menu called Safari > Preferences > Privacy > look for the Apple Pay section. 
    2. Note: It can take about 5-10 minutes to propagate after you sign-in to iCloud.
  4. Done! Any website or app with Apple Pay enabled for checkouts will render an Apple Pay button as a checkout option. To test this you can go to Stripe’s Apple Pay website and test it out. (Don’t worry purchases happen in their test environment.)
  5. There are a few caveats:
    1. This only works in Safari
    2. You can only complete the purchase if your iPhone is connected to your Mac. Apple Pay requires TouchID to authorize the purchase.

Oh and if this article worked for you please consider sharing it or buying me coffee!