Installing ChromeDriver on Windows

Installing on Windows 7 & 10:

The following are the best ways to install ChromeDriver on Windows so you can run Selenium:

  • The easiest way to install ChromeDriver is to use a package manager such as Chocolatey. You should really have a package manager if you don’t now:
    1. Install Chocolatey. I used the cmd.exe method and it worked really quickly.
    2. Install ChromeDriver choco install chromedriver
    3. Confirm it was installed by typing chromedriver —version into your command line and seeing that it returns a version number. If it errors, ChromeDriver wasn’t installed.
  • Or run Chrome & ChromeDriver in a container such as Docker. Simply download the combined container, start it and point your code at the right address. Here’s an example.

Which methods have you found the easiest or most success with? Which methods didn’t work for you? Please leave a comment below. 


If this article worked for you please consider following me on twitter, sharing it or buying me coffee!

How To Run Your Selenium Tests with Headless Chrome

The Problem

If you want to run your tests headlessly on a Continuous Integration (CI) server you’ll quickly realize that you can’t with an out-of-the-box setup since there is no display output for the browser to launch in. You could use a third party library like Xvfb (tip 38) or PhantomJS (tip 46) but those can be hard to install and aren’t guaranteed to be supported in the long run (like PhantomJS).

A Solution

Enter Headless Chrome. Headless is simply a mode you can put Chrome into that allows it to run without displaying on screen but still gets you the same great results. This is a better option than using Chrome in a Headless manner such as in a docker container where the the container actually uses Xvfb.

Starting with Chrome 59 (Chrome 60 for Windows) we can simply pass Chrome a few configuration options to enable headless mode.

An Example in Ruby

Before we start make sure you’ve at least got the latest version of Chrome installed along with the latest version of ChromeDriver.

Let’s create a simple Selenium script (the example is posted below).

  1. We will pull in the requisite libraries and then create our setup method where we will pass Chrome our headless option as a command line argument. The first add_argument of ‘– headless’ allows us to run Chrome in headless mode. The second argument is, according to Google, temporarily required to work around a few known bugs. The third argument is optional but gives us the ability to debug our application in another browser if we need to (using localhost:9222).
  2. Now let’s finish our test by creating our teardown and run methods:

Here we are loading a page, asserting on the title (to make sure we are in the right place), and taking a screenshot to make sure our headless setup is working correctly. Here’s what our screenshot looks like:

headless

Expected Behavior

When we save our file and run it (e.g. ruby headless_chrome.rb) here is what will happen:

  1. An empty chrome browser instance will open
  2. Test runs and captures a screenshot
  3. Browser closes

Outro

Hopefully this tip has helped you get your tests running smoothly locally or on your CI Server. Happy Testing!

Additional References

This was originally written for and posted on the Elemental Selenium newsletter. I liked it so much I decided to cross post it with some updates.

Oh and if this article worked for you please consider sharing it or buying me coffee!

How To Run Your Selenium Tests Headlessly in Docker

The Old Way

It used to be that in order to get your Selenium tests running on a given machine you had to install each individual browser and then the browser drivers (for instance ChromeDriver for Chrome). Some of my most popular posts are about installing these drivers. However that’s all changed if we instead use containers. Simply download and run a single Docker container with both the the browser and the driver pre-installed.

Problem solved!

…Except now we’ll be running your tests headlessly which customers don’t do.

The Catch: Headless Browsers

In the context running a Selenium test headlessly in Chrome, it means Chrome launches and runs the test in the browser but you don’t see it because there is no GUI. Instead you get a command line to run and debug your tests. The downside is one customers don’t use your application in a headless state and two these tests are harder to debug.

The upside is we can manage the installation and running of the browsers and drivers in an easier way. We can also run our tests on devices that don’t have computer graphics like AWS EC2 instances (virtual machines), and CI/CD services like TravisCI and CircleCI to name a few.

Docker Desktop and Container Installation:

In this exercise we will download a single (standalone) Chrome browser with ChromeDriver already configured. Then we will simply run a test!

  • First thing we need to do is install Docker Desktop for our respective OS (Mac or Windows)
    • Once this is up and running we can download our container
    • (Note: this is a desktop app but can also be installed from the command line for future virtual machine use)
  • Second, since we already know what container we want, we simply need to copy and paste this command into our terminal docker pull selenium/standalone-chrome
    • This will download the Chrome container image
  • Third, we run is to start the container so Chrome is running
    • We can run docker ps at any time to see if any containers are running
    • We now run
      docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome
    • This sets the docker container to use the port 4444. We will use this later in our test file.

Running Our Tests: An Example

Now we’re ready to write a test using Selenium and our now running Docker container with Chrome in it. For fun I’m going to write this in Ruby but it should work for any language:

Expected Behavior

When we save our file and run it (e.g. ruby headless_chrome.rb) here is what will happen:

  • A chrome browser session will open within the docker container
  • Test runs and saves a screenshot
  • Browser closes
  • All we see is the result of the test

There you go! We managed to install a container with both Chrome and ChromeDriver installed on it and then wrote and ran a test to demonstrate it works. You’ve now run a Selenium Test Headlessly with Docker!

In the larger picture this means each time we need to install Selenium on a new computer (CI, EC2 instance, virtual machine, desktop) it’s simply about installing docker and downloading an image, spinning it up and connecting it to your test. After having done this half a dozen times I think it has huge advantages over the old ways (link to Installing ChromeDriver on macOS).

References:

  • Selenium Docker Hub for finding the approprite container
  • This is not the same as using Headless Chrome (which is a configuration option in modern versions of Chrome). To learn more about using Headless Chrome go here.

 

Oh and if this article worked for you please consider sharing it or buying me coffee!

8 Tools I use to Accelerate My Testing

Inspired by Justin Rorhman’s post of a similar name with a slight twist focusing on tools that generally help accelerate my testing.

As a test and quality specialist embedded in an engineering team I have a lot of work to do on any given day. Our engineering team’s goal is to ship quality software on a regular basis to deliver value to our customers. Naturally I rely heavily on a number of tools to help me better understand what is going on with the software or code under test and/or to accelerate what I’m able to do in a given amount of time.

Like Justin said in his article “Testing tools don’t make me a good tester, but they do make me a better tester. These tools help amplify things I need to do repeatedly or give me access to the product I wouldn’t have otherwise. They also shape the way I think about testing problems, for better and worse.”

In no particular order these tools are:

  1. GitHub. We use the GitHub workflow to deliver all of our code including application, library, and test automation. With each push to a particular branch our CI system builds the application and runs our unit and integration tests. With each Pull Request I can see all or just specific changes, code reviews and test run summaries. All this helps keep our build pipeline clean, makes it easier to identify potential issues quickly and helps me quickly build my test strategy.
  2. Docker. Each separate application of our system has been dockerized (aka runs in a container). This makes it incredibly easy to set new people up (a few hours of setup instead of a day) and gives us some powerful logging for each application. It also reduces conflicts when switching between branches where libraries or decencies have been changed.
  3. Heroku review apps. As I mentioned above with each push our application is compiled through our CI system. Once finished our CI system pushes that complied build to Heroku which automatically provisions a staging like environment accessible through its own url. (This feature is called a Heroku Review application). In about 5 minutes after a set of changes have been built our team now has a brand-new test environment ready for those hard to test local problems like testing on real mobile devices or a few special integrated services. Being able to build these new automatic quickly allows our developers to show off / get quicker input to the work they are doing.
  4. Browser DeveloperTools. I primarily use Chrome and Safari but the developer tools in each browser are priceless. From Inspecting requests and looking at their data, to setting or removing experiment variables, debugging problems, looking for problems, profiling, responsive design, layout, checking DOM elements, etc browser DeveloperTools really are the first place to start. It would be so much harder to learn what is going on without them.
  5. CircleCI. Great for keeping your deployment pipeline clean, running your build tests and ensuring those changes get where they are going each time. It doesn’t much matter what CI system you use it’s helping to accelerate something that would normally take away from developer or tester time.
  6. EverNote. Surprising or not, I’m always taking notes about any number of things like quick terminal commands, operations for creating data, migrating databases in heroku, setting up configurations, test ideas, questions for follow up, etc. Anything I might forget goes here and I can get to it anywhere, any place on any device.
  7. iTerm2. A command line replacement utility with Oh My Zsh installed that I’ve customized and continue to customize with shortcuts and aliases to make my repetitive tasks quicker. Also customized with visuals showing which repo I’m in, what branch I’m on and if there are uncommitted files, because I do enough on the command line.
  8. Automated UI acceptance tests. Running our acceptance tests for each staging deploy (or set of deploys) allows us time to do more deep exploration on the application as a whole with a reduced focus on those areas covered by the existing tests.

Part of the desire for writing this was to record what my work looks like now and then be able to compare it later-on in the future. However, in the time since I started writing this article I changed jobs and now the future is here. Well, kind of. My new team’s development stack is a lot more varied, which means so are the tools I’m using. I think it’s still best to wait for half a year or so before I write a follow up post on the tools accelerate that work. In the meantime, what tools do you use that accelerate your testing?

Good and Bad UI Test Automation explained – Inspired by Richard Bradshaw’s Tweets

Generally speaking there’s a scary trend with the influx of people interested in test automation where (seemingly) everyone wants to automate at the UI level. For example the phrase “Test automation” seems to be synonymous with UI automation which seems to mean using Selenium. To be fair there are numerous reasons for this. First, if you primarily focus on functional testing through the UI, this application of automation is the most relevant and straightforward to understand. Second, there are a lot of resources out there (books, blogs, classes) for Selenium. Third it is much easier to write some blackbox automation at the UI level then to develop competence at writing automation at the API or unit level. (You should still learn to do this lower.)

So Richard took to Twitter to explain his thoughts. Let’s take a look:

Before we go very far, let’s look at what Richard means by targeted tests.

Tests are “targeted” if they are designed to reveal a specific risk or failure (we might call this risk-based). If you are doing risk-based testing, your tests should ALWAYS be designed this way, regardless of how you intend to execute the tests (automated or not).

Seams are how you use the system’s implementation to help you build more streamlined and realiable automated tests. Seams can be can be APIs, mocks, stubs, or faked dependecies of your real system.

Want some examples of Seams? Checkout this blog post by Angie Jones.

Yes, most times we can use better designed tests! Want to test the various ways to get to a particular feature? Use PATH testing. Want to test the feature itself? Use functional or domain testing, etc. Some test techniques will be better at surfacing one type of problem and bad at surfacing another. It’s not always necessary to design your UI tests as long feature rich Scenarios.

When we talk of interfaces and automating at the lowest level it is usually a good time to mention this is what the Test Automation Pyramid tries to describe. There are lots of versions of this but I recommend Alan Page’s Test Automation Snowman.

If you can find the failure by writing a test lower in the stack (at the API level intead of UI level or at the unit level instead at the API level), the faster it runs and the faster you get feedback. If you do have to write the test at a higher level like the UI can you find some way to decrease the setup time / data of the test? (See above for Angie Jone’s blog post.) If you aren’t sure maybe thinking about whether you are testing the UI or testing through the UI will help?

Yes! Adding testability like IDs, classes, data-test attributes, setting up test data, mocking, etc. should be done as quickly and dependbly as possible. If you can’t do this on your own (or you have a different process for making changes) bring it up during sprint planning in order to get the development teams help behind building a solution.

This IS a thing. Selenium is great, it’s open source and the WebDriver protocol is the soon-to-be de facto standard for interacting with browsers. However there are lots of tools, some will use Selenium under the hood and others that are completely different (think Cypress.io). Depending on the problem you need to solve, you may find a specific tool much better than Selenium. GitHub is a really good resource for finding open source tools and when you find something that might work, give it a try.

If you want to write Good UI Test Automation your tests should be ideally suited for the UI (and not some other layer), targeted, reliable, use seams where possible andcover a specific risk. If you aren’t here yet work to refactor your test code until you can get there.

Did I miss anything about Richard’s tweet storm that was important?

Practice using Selenium Now!

Have you ever wanted to learn a little bit about Selenium WebDriver but didn’t know where to start?

Turns out there are some good tips / tutorials online for practicing writing Selenium in Ruby. One of those is a newsletter called Elemental Selenium that has something like 70 tips. You can sign up for the newsletter if you want but what I found valuable was to look at several of these tips, write them out (don’t copy + paste) and make sure you understand what they do. Turns out when you do this and you commit them to a repo, you can reference back to them when you come across similar problems in the future.

Simply stated [highlight]the goal is to:[/highlight]

  1. Read through the Elemental Selenium tips and then write (don’t copy + paste) the code yourself.
  2. Try running the tests locally and see how things work.
  3. Once you’ve written a few tests, refactor those example tests so they become more DRY (don’t repeat yourself). Create page objects.
  4. Commit these to your own repo on GitHub.

Putting your code on GitHub will have the benefit of showing you can write (at least) basic selenium automation. Although this code may not be your “best foot forward” given how new you’ll be, it is a starting point. As you learn more and make improvements, your code will reflect this.

These tips are (hopefully) grouped correctly but within each group there may be some variance. See if you can do one or two per day (some will be easier than others). If you see something interesting and want to jump to it immediately, go for it.

Beginning to Intermediate:

Intermediate to Advanced:

I’ve recommend a few people try this exercise because I found it valuable. Am I missing anything else? Has anyone else done something similar but in a different language or tool?

Installing GeckoDriver on macOS

Overview of naming conventions

  • GeckoDriver is the library you need to download to be able to use Selenium WebDriver with Firefox. These are the Selenium Bindings.
  • Marionette is the protocol which Firefox uses to communicate with GeckoDriver. Installed by default with Firefox.
  • FirefoxDriver is the former name of GeckoDriver.

Ways to install GeckoDriver:

  1. The easiest way to install GeckoDriver is to use a package manager like brew or npm such as npm install geckodriver. This method requires you some package manager installed but you probably should anyways.
  2. Run Firefox and GeckoDriver in a container using Docker. Simply download the combined container, start it and point your code at the right address. I’ve written about how to do this using Chrome, should be very similar to do Firefox.
  3. Specify it in your Selenium setup code. If you go this route, you can include ChromeDriver as well.
  4. Download the driver and add its location to your System PATH. These instructions are for Chrome but should work for GeckoDriver as well.

Once this is done, it should work like nothing has changed. The big advantage is you’ll now be able to use Firefox browsers newer than 46!

Selenium-WebDriver 2.53.x not working with Firefox 47 and beyond

The problem

I’m used to running selenium tests against Firefox locally (OS X Yosemite and now MacOS Sierra) both from the command line using RSpec and when using a REPL like IRB or Pry. I don’t use FF often so when I started having issues I couldn’t remember how long it had been since things were working. The problem was pretty obvious. The browser would launch, wouldn’t accept any commands or respond to Selenium and then error out with the message:

Selenium::WebDriver::Error::WebDriverError: unable to obtain stable firefox connection in 60 seconds (127.0.0.1:7055) from /usr/local/rvm/gems/ruby-2.1.2/gems/selenium-webdriver-2.53.0/lib/selenium/webdriver/firefox/launcher.rb:90:in `connect_until_stable’

This occurred for Selenium-WebDriver versions 2.53.0 and 2.53.4. It also seemed to occur for Firefox versions 47, 48, and 49.

The solution

Downgrade to Firefox 45.0 Extended Service Release (ESR).

I’m not the first one to post about the upcoming changes and lack of support for Firefox 47+. I probably deserve the error message for not paying more attention to the upcoming changes and will certainly look forward to implementing the new MarionetteDriver.

Installing SafariDriver on macOS

Safari + WebDriver aka SafariDriver comes included in Safari 10 which means as long as you have Safari 10 and later versions you can point your tests at Safari and run them without installing anything else.

Safari now provides native support for the WebDriver API. Starting with Safari 10 on OS X El Capitan and macOS Sierra, Safari comes bundled with a new driver implementation that’s maintained by the Web Developer Experience team at Apple. Safari’s driver is launchable via the /usr/bin/safaridriver executable, and most client libraries provided by Selenium will automatically launch the driver this way without further configuration. – WebKit Announcement 

That’s right, no brew installs and no system PATH setups!

Additional References:

Debugging Selenium code with IRB

Occasionally something will change in our system under test that breaks a Selenium test (or two). Most of the time we can walk through the failure, make some tweaks and run it again – repeating the process until it passes. Depending on how long it’s been since we last worked with the code, or how deeply buried the code is, it may not be enough to fix the test and we may have to tackle one or more of the underlying methods we used to build the code.

In these situations it can be helpful to debug our tests using an interactive prompt or REPL. In the case of Ruby we can use irb or Interactive Ruby to manually step through the Selenium actions, watching and learning. Here’s the general format for working with Selenium in irb; it’s very similar to how we code it:

  1. Launch interactive ruby: irb
  2. First, import the Selenium library: require 'selenium-webdriver'
  3. Second, create an instance of Selenium and launch the browser: driver = Selenium::WebDriver.for :firefox
    1. If you’ve got Chrome installed with an updated PATH, you can also swap firefox for chrome.
  4. Third, start entering your Selenium commands: driver.get 'http://www.google.com'

Here’s a simple example using irb:

In this example I’m bringing up Chrome as my browser and navigating to Google. I’m finding the search query box, typing my last name ‘Kenst’ and clicking enter. Thus an example of searching Google for my last name!

Technically once the browser is up we can navigate to whatever page we need without typing ALL of the individual commands. This is really valuable in those instances when you need to login, then navigate several pages before getting to the place you can debug. In other words do all the setup outside of irb, directly in the browser. Once you are in the proper location step through your code one command at a time (and lines with => show the responses to our commands, if any). Those responses will help us debug our tests by confirming what elements Selenium is picking up and what our methods are returning (if anything).

Additional References: