Appropriate Test Documentation & Formatting

The Question

Recently in an online forum a tester person asked:

Does someone have a simple example of test case (excel sheet) format? I am the only one tester in my company and we are trying to arrange the test documentation. Any advice or example will be useful.

I wish someone had given me this advice when I first started out:

Be careful about using someone else’s formatting and/or templates. If you don’t know what you are doing, you might be inheriting their issues and misunderstandings. It’s usually best to understand what you want to accomplish and then find a solution for yourself. Especially if it’s something simple like using Excel for documentation. Tables and matrices are great for organizing tests and don’t cost much money or time to make.

Appropriate Test Documentation

The appropriate documentation for your tests (or test cases) will depend to a large degree on the test technique and approach you use. Test Case Management systems (and similar systems) are often designed for functional or scenarios tests where the format is similar and you can list out multiple steps. Other test techniques such as Path Analysis, Domain Testing and even Risk Based testing can look very different when they are well designed. The same can be said when you decide to take an Exploratory approach to your testing where you are likely to write Exploratory Charters. Cramming all of these different techniques into one system usually isn’t the best idea.

The Smallest Amount

Focus on the smallest amount of documentation you need to do your job well. You have a finite amount of time and if you spend too much of it creating documentation you will have less time to do the work. If you are writing tests for other humans to read and understand, they should be written in a way that communicates this well such as using visuals (mind maps, diagrams, etc.) and summaries. Similarly if you are writing the tests for computers to read and understand they can (need to be) much more granular and step by step.

I’d wager most experienced engineers have gone through this process of: trying to document as much as possible but by the time you are done you didn’t do enough testing and the documentation isn’t done either. Maybe you put it all into a test case management system that no one else cares to read because it’s in this weird format that probably only makes sense to you.

Staying Agile

Focusing on the appropriate documentation for your tests and then creating the smallest amount you need to do your job well also allows you to have agility. Agile is all about being able to adjust to changes as they come in. Quick changes in direction aren’t kind on large amounts of pre-scripted documentation and/or can result in lost work.

I wish this approach was more obvious but a quick google search will reveal it’s not the dominate theory. However, if you read this and take the advice you’ll be learning from my mistakes. Focus on using the smallest amount of documentation you need to be successful and making sure that documentation is appropriate for your test choices will give you agility and future proofing even as you scale your team size and move into the future.

How To Run Your Selenium Tests Headlessly in Docker

The Old Way

It used to be that in order to get your Selenium tests running on a given machine you had to install each individual browser and then the browser drivers (for instance ChromeDriver for Chrome). Some of my most popular posts are about installing these drivers. However that’s all changed if we instead use containers. Simply download and run a single Docker container with both the the browser and the driver pre-installed.

Problem solved!

…Except now we’ll be running your tests headlessly which customers don’t do.

The Catch: Headless Browsers

In the context running a Selenium test headlessly in Chrome, it means Chrome launches and runs the test in the browser but you don’t see it because there is no GUI. Instead you get a command line to run and debug your tests. The downside is one customers don’t use your application in a headless state and two these tests are harder to debug.

The upside is we can manage the installation and running of the browsers and drivers in an easier way. We can also run our tests on devices that don’t have computer graphics like AWS EC2 instances (virtual machines), and CI/CD services like TravisCI and CircleCI to name a few.

Docker Desktop and Container Installation:

In this exercise we will download a single (standalone) Chrome browser with ChromeDriver already configured. Then we will simply run a test!

  • First thing we need to do is install Docker Desktop for our respective OS (Mac or Windows)
    • Once this is up and running we can download our container
    • (Note: this is a desktop app but can also be installed from the command line for future virtual machine use)
  • Second, since we already know what container we want, we simply need to copy and paste this command into our terminal docker pull selenium/standalone-chrome
    • This will download the Chrome container image
  • Third, we run is to start the container so Chrome is running
    • We can run docker ps at any time to see if any containers are running
    • We now run
      docker run -d -p 4444:4444 -v /dev/shm:/dev/shm selenium/standalone-chrome
    • This sets the docker container to use the port 4444. We will use this later in our test file.

Running Our Tests: An Example

Now we’re ready to write a test using Selenium and our now running Docker container with Chrome in it. For fun I’m going to write this in Ruby but it should work for any language:

Expected Behavior

When we save our file and run it (e.g. ruby headless_chrome.rb) here is what will happen:

  • A chrome browser session will open within the docker container
  • Test runs and saves a screenshot
  • Browser closes
  • All we see is the result of the test

There you go! We managed to install a container with both Chrome and ChromeDriver installed on it and then wrote and ran a test to demonstrate it works. You’ve now run a Selenium Test Headlessly with Docker!

In the larger picture this means each time we need to install Selenium on a new computer (CI, EC2 instance, virtual machine, desktop) it’s simply about installing docker and downloading an image, spinning it up and connecting it to your test. After having done this half a dozen times I think it has huge advantages over the old ways (link to Installing ChromeDriver on macOS).

References:

  • Selenium Docker Hub for finding the approprite container
  • This is not the same as using Headless Chrome (which is a configuration option in modern versions of Chrome). To learn more about using Headless Chrome go here.

 

Oh and if this article worked for you please consider sharing it or buying me coffee!