18 GitHub Projects for Testing

Aside from it’s many awesome lists GitHub is a really good place for open source testing tools, libraries and frameworks (and their corresponding code). I’m pleasantly surprised by these new (and sometimes old) testing resources, so I’d like to highlight many of them in the hopes others might also find them useful.

The challenge with listing any GitHub projects is that there are really too many projects to list. I’ll push this problem and mention a few here anyways. Many of these projects are Ruby based because that’s my language of choice. I’m sure there are similar projects for your favorite languages as well.

Tools

Many of these tools are easy to download or install without any knowledge of their underlying code. A few like The Internet or Ally will require a little more work.

  • BugMagnet. Both a Chrome and Firefox extension that contains lists of values you can use for boundary testing. Great exploratory testing companion.
  • Form Filler. Other Chrome extension is great for filling in standard forms (login pages, etc).
  • The Internet. A test project for playing with selenium automation but could probably be cloned and used for other testing / practice purposes.
  • AutoHotkey. A windows desktop automation project for helping you speed up your workflow.
  • Android battery historian. Inspect battery related information and events on Android devices.
  • PICT. A pairwise / combinatorial tool for Windows from Microsoft.
  • Ally. An accessibility auditing tool.

Libraries

These offer extended functionality, generate data or generally make testing easier.

  • Faker. This great gem will help you generate fake data for testing. Need passwords, usernames, email address, city information? Faker has it.
  • Parallel Tests. This gem is for running multiple tests at one time. Personally I use this for running concurrent automated tests locally and at Sauce Labs.
  • Mailosaur Ruby Client. This gem is part of a mailosaur service but in theory will help with end to end automated testing around mocking email services.
  • BrowserMob Proxy. A proxy for monitoring and manipulating traffic. ElementalSelenium has a few tips showing how to use BrowserMob in your test automation.

Frameworks:

These are pretty well known across the testing landscape and for that reason are worth mentioning. Many of them have good wikis, documentation or other related resources you can learn from in addition to the underlying code and bug trackers. YMMV:

My goal with sharing these is to point on the many very good tools available for testing and I hope I’ve accomplished this. Did I miss any projects that you find useful for testing? If so leave a comment and tell me what you like about the particular projects!

9 GitHub Lists for Testing

I spend a lot of time on GitHub and it can be a great place for finding open source libraries, tools, frameworks and pretty much anything else you might want to version control. This includes lists (and more often than not, lists of lists). The challenge is finding just those lists that contain value and not chasing around each individual list of list in a recursive never ending search.

Why Lists?

Over time and when looking for certain types of failures (or bugs), patterns emerge. Some of these patterns can be captured as data, like password dictionaries or image catalogs, or as a collection of test ideas. Some authors have made lists this with heuristics (Bach, Bolton & Hendrickson) while others have published lists of failures common to certain applications or languages (Kaner).

Here are 9 lists I’ve found to be good references when testing.

Input Fields:

Useful for boundary analysis and equivalence class partitioning, input field catalogs are basically collections of values you can use to try to trigger failures based on the data-type of the input field. For this reason they are often broken into specific data types like Strings, Integers, Floating Points, etc.

Security:

For both tools and reference material, turns out there are some good references for learning more about security. Password libraries and other data:

More:

Other valuable lists that don’t easily fall into a single category:

  • My own catalog of images. Based on size and format you can use this catalog for testing image uploads. Searching Google I wasn’t able to find any specific collection for testing, so I made my own.
  • Awesome Test Automation. A curated list of test automation frameworks, tools, libraries, etc. The list is pretty good. I use Ruby and they had a good list of Ruby gems for generating test data.
  • TestingConferences.org. A simple list of software testing conferences and workshops published collaboratively with the testing community.
  • Free Software Testing Books. That’s right, a collection of free software testing books. Although some of these appear to be papers, guides and “demo” chapters, it’s still a good (cough, free) reference!

Did I miss any lists that you find useful for testing? If so leave a comment and tell me what you like about the particular list!

Practice using Selenium Now!

Have you ever wanted to learn a little bit about Selenium WebDriver but didn’t know where to start?

Turns out there are some good tips / tutorials online for practicing writing Selenium in Ruby. One of those is a newsletter called Elemental Selenium that has something like 70 tips. You can sign up for the newsletter if you want but what I found valuable was to look at several of these tips, write them out (don’t copy + paste) and make sure you understand what they do. Turns out when you do this and you commit them to a repo, you can reference back to them when you come across similar problems in the future.

Simply stated [highlight]the goal is to:[/highlight]

  1. Read through the Elemental Selenium tips and then write (don’t copy + paste) the code yourself.
  2. Try running the tests locally and see how things work.
  3. Once you’ve written a few tests, refactor those example tests so they become more DRY (don’t repeat yourself). Create page objects.
  4. Commit these to your own repo on GitHub.

Putting your code on GitHub will have the benefit of showing you can write (at least) basic selenium automation. Although this code may not be your “best foot forward” given how new you’ll be, it is a starting point. As you learn more and make improvements, your code will reflect this.

These tips are (hopefully) grouped correctly but within each group there may be some variance. See if you can do one or two per day (some will be easier than others). If you see something interesting and want to jump to it immediately, go for it.

Beginning to Intermediate:

Intermediate to Advanced:

I’ve recommend a few people try this exercise because I found it valuable. Am I missing anything else? Has anyone else done something similar but in a different language or tool?

The Anti-Library

The writer Umberto Eco belongs to that small class of scholars who are encyclopedic, insightful, and nondull. He is the owner of a large personal library (containing thirty thousand books), and separates visitors into two categories: those who react with “Wow! Signore, professore dottore Eco, what a library you have! How many of these books have you read?” and the others – a very small minority – who get the point that a private library is not an ego-boosting appendage but a research tool. Read books are far less valuable than unread ones. The library should contain as much of what you don’t know as your financial means, mortgage rates and the currently tight real-estate market allows you to put there. You will accumulate more knowledge and more books as you grow older, and the growing number of unread books on the shelves will look at you menacingly. Indeed, the more you know, the larger the rows of unread books. Let us call this collection of unread books an antilibrary.

Nassim Nicholas Taleb in the Black Swan

A Collection of Menacing Books

The more I attend conferences, interact with others and read blogs in the various software communities I take interest in (testing, entrepreneurship, technology, etc.) the greater the number of books I find I’d like to consume. It seems almost automatic these days to ask for books or references someone has found useful to solve problems or gain a wider (or deeper) perspective on a subject.

Yet a collection of books on a list or sitting on in Dropbox isn’t nearly as menacing and doesn’t call to me the same way as having those books sitting on my shelf where I have to watch them age.

No Clutter
I try to find balance between only owning those things I can and do use regularly and a more consumerist desire where I need to have lots of options. I’m more on the minimalist side where I want no clutter (of anything) so I constantly edit what I own. Part of this comes from wanting to be practical and trying to understand trade-offs between buying something now or something better later. Part comes from the fear of getting “behind” or having too many choices.

This manifests in many ways including trying not to accumulate too many books.

A Research Tool

When I first toured Cem Kaner’s personal library I fell into the first category of people from the quote above. I think I said, “Wow what a library”, as the size of his personal collection was more immense than anything I’d seen. Now it wasn’t comparable in size to Umberto Eco’s but it was impressive enough that I felt the need to tell him so. My parents have book shelves filled with books they read twenty years ago but Cem had bookshelves full of books he hadn’t read and those he kept around for reference. After my “impressive” comment, Cem made it clear the point of his private library was not built to impress but as a research tool.

Keeping materials around for research or reference makes sense. Some you might keep around because you feel a special attachment to or because they gain in value but most books don’t. They aren’t trophies either.

The Anti-Library

The anti-library (or antilibrary) seems like the logical compromise between acquisition and usefulness. Get rid of those books you’ve read and aren’t likely to reference again and keep the collection of books you haven’t. Don’t make it your library, make it your anti-library.

I’m an AST BBST Instructor

That’s right, I completed the trifecta of BBST classes and decided to continue on to being an Instructor. I finished the class in November but just recently got the proof:

Boom! Now comes the hard work – working as an assistant through enough classes until I learn the system and work my way up to a lead instructor role for AST.

Part of my desire for taking the BBST Instructor class is to become more familiar with material taught in the BBST classes, get exposed to new student’s ideas (that will hopefully broaden and challenge my understanding) and learn how to provide important and useful feedback (and help others do the same thing).

Eventually I hope to teach the BBST materials to people I work with and help expose others to the material outside of AST. This was the first step.

I’m a Black Box Software Tester

More accurately I should say I’ve passed the Association for Software Testing‘s Black Box Software Testing Foundations class.Here’s the proof:

What does this mean? It means the instructors think I understood enough of the material to pass me based on the work I did throughout the course which includes discussion forums, assignments, quizzes and a final examination. I feel like I learned a lot from the exercises, readings and watching Cem Kaner’s videos.

For those who don’t know, Foundations is an intense 4 week class covering the basics of black box testing including the mission of testing, the oracle problem, the measurement problem and the impossibility of complete testing. I’d definitely recommend the class as long as you can spare at least 12+ hours a week to commit to watching the videos, reading the required and recommended readings and participating in the assignments. You must take the Foundations course before you can take any further classes so you learn how they are run.

One of my goals for 2013 is to take the other courses: Bug Advocacy, Test Design and then Instructor!

A Test Group of One

I started the Foundations Black Box Software Testing class earlier this week and we are now into lesson two. I thought the discussion question was interesting so I’d like to share it along with my response:

[P]lease describe the role of the test group (services and responsibilities) in your organization. How do you think this mix differs from what you think of as the “typical” test group? How would you change this?

My response

Up until a month ago I was the sole tester in small software company (our entire company was about 11 people). Since then we’ve been purchased by a large multinational company with over 15,000 people world-wide. I’m still not quite sure what exact group I belong to or what my overall responsibilities are so I’ll focus my answer on the company prior to this acquisition.

Since our company was small so was our technical team, which consisted of 3 full time developers and me. For a few years we also had a CTO who acted as a development manager and occasionally for big projects we’d hire contractors. Like others on our technical team I essentially had two roles in the company: first as a tester of our products and secondly as a jack of all trades, used where-ever I’m needed.

My responsibilities and services include:

  • Support our existing Clients deployments
    • Questions about software behavior 
    • Debug problems with existing systems
  • Install and Configure customer deployments (production systems) 
  • Install and Configure test and development labs
  • Compile / build the releases for our software (release management)
  • Document installation procedures and update existing online help system 
  • Test software releases and patches which includes:
    • Make sense of the changes development made 
    • Understand what was asked of our development team (no formal specs) by our customers or executives 
    • Make sure the company knows the risks and stability of the release in both functional and para-functional areas. 
  • Anything else asked of me by management 

While these responsibilities can limit the amount of time I can spend on a particular release testing, they provide different points of views which feed into what I know and learn about the product. Those influences then get reflected in my testing approach. This holistic view can help identify pain points which I might otherwise not get to see.

I’ve worked in large and small companies. Each organization seems to have services and responsibilities that suit what someone thinks the companies needs are. I think the difference between mine and other testing groups has been the amount of time devoted to learning about the craft of software testing and how that can be applied to helping the business. The only way to change this is to set a good example for others to follow.

Enrolled in BBST Foundations

It’s official. I’m enrolled in the BBST Foundations course for November through AST.

I joined AST (Association for Software Testing) with the end-goal to enroll in the BBST (Black Box Software Testing) Foundations course. I’ve read about the classes, seen a number of experts whom I trust recommend them and also heard good things from my post on SQA StackExchange. BBST.info which is home to Cem Kaner and Rebecca Fiedler (the creators of the material) BBST consulting practice say this:

Too many testing courses emphasize a superficial knowledge of basic ideas. This makes things easy for novices and reassures some practitioners that they understand the field. However, it’s not deep enough to help students apply what they learn to their day-to-day work.

The BBST series will attempt to foster a deeper level of learning by giving students more opportunities to practice, discuss, and evaluate what they are learning. The specific learning objectives will vary from course to course (each course will describe its own learning objectives).

The Black Box Software Testing course information is all freely available online at TestingEducation.org but it seems like I’d get more information going through the class with people in a more collaborative environment. More importantly it’s taught by teachers who have been trained by Cem Kaner and Rebecca Fiedler themselves.

There are several levels to the BBST coursework including:

  • Foundations
  • Bug Advocacy
  • Test Design
  • and eventually… Instructor
Just the list of required and recommended readings at the bottom of the BBST Foundations website has me excited. It’s going to be a lot of work but probably well worth it.
In the video below Dee Ann Pizzica explains the BBST courses at CAST 2011 (Conference of the Association for Software Testing). It’s worth a watch:

Do Software Testers Need a College Education

I came across an old blog post on the uTest blog Do Software Testers Need a College Education. The author says:

Depending on who you ask this question to, you’re likely to receive various degrees (pardon the pun) of yes and no. Or you may find many others who answer in a noncommittal way: “it depends.”

Having worked closely with thousands of software testers in the uTest community, I can attest to the fact that many testers do in fact have impressive resumes with regard to higher education (master’s degrees, PhD.s, etc.). However, there is also convincing evidence that demonstrates quite the opposite. So if you let the data speak for itself, what is one to believe?

The article goes on to list a few explanations from the For, Against and It depends camps. I was trying to comment on the article but it wasn’t working. If they turned off the comments for this article then why is the comment box still available? I must have found a bug!

Here is my input:


You don’t need a college education to learn technical knowledge / skill, just time and desire. You might be able to make the argument a college education makes Testers and Developers better but it would depend on the school, the person’s learning habits and how much they learned. I’ve seen quite a few reports that indicate Software Engineering (which includes Testers) has the highest concentration of non-college educated engineers in all of the engineering disciplines. This might to suggest the industry is more open to taking chances. It might also suggest if you are qualified it, have the technical skills, it doesn’t really matter how you learned as long as you can do the work.

Those hiring might prefer employees to be college educated because it’s a safe bet. Especially for those who don’t know how to hire, have an HR dept who screens candidates, etc. Rather I believe most companies are better off with someone with relevant experience and the desire to learn more. I learned everything I know about Software Testing after I graduated from college. Today there are hardly any colleges that offer classes or degrees in software testing and plenty of scammers trying to offer Certification as a replacement for learning to test.

Luckily there are people like James Bach and Cem Kaner who try to help train testers.

Check out what James thinks software testers need to learn.

Anyone Can Test, right?

In this video from StarEast Rob Sabourin talks about his experience with concept of “anyone can test”. This actually gives me an idea for a challenge:

Before watching, make a list or jot down some notes that try to describe the potential benefits and problems with assigning any particular person to test your product or service. Finally what the video and do a comparison. How many things did you come up with vs. Rob. Then formulate your own response to can anyone test?

Anyone can test, right?