Getting Paid to Learn

Picking up experience on the job can be great (for example learning about SEO, application monitoring or observability) but all things being equal, it shouldn’t come at the expense of the many other ways we learn. Without the support to continuously learn new things we get stuck in jobs and become less valuable to the market and our employers over time. That’s why I’m a big fan of the concept of Getting Paid to Learn.

Getting Paid to Learn

Just like it sounds, I’m talking about companies that help and support their employees learn and develop new skills over time by giving them time and budget to learn new things.

Getting the Time

I joined BloomNation over a year ago and began learning how to write code in a brand new language: JavaScript. My boss knew this meant giving me time to learn and time to produce something at a far slower rate than I might normally be able to. (Note: this is time during the regular work day) The upside of this meant alignment with my front-end developers and now I have some skill that I can use again and again in other areas like in helping to do front-end Code Reviews.

Getting the Budget

Years ago when I first started learning how to write UI automation I requested my then company buy me an online book. As I used my new skill to tackle newer and larger problems I outgrew the advice a book could give me and asked for the budget to hire a known consultant to help guide me in more advanced practices. We set a budget and hired that consultant part time.

While you don’t necessarily have to hire a consultant, having a defined (or loosely defined) annual budget per person for gaining education through online courses, books (FYI anyone can gift you a Kindle eBook fairly easily), conferences, is a huge signal that the company values you, is invested in your growth and its also a pretty nice recruitment tool.

Peace of Mind

My own experience tells me companies are starting to get the hint about having an active policy toward continuous learning. Some have enterprise training accounts with Udemy, Udacity, Safari Online or other programs. Others have a defined budget per year. This has become one of my goto questions when I evaluate a company to join: do they offer both budget and time to learn new things?

While I may not always have the time to take advantage, when I do want to learn something new, upskill or generally increase my depth of knowledge I want the safety and confidence of knowing the company is willing to pay me to learn.

You don’t have to be an expert to teach

You don’t have to be an expert to teach someone else, you just need to know a little more than they do.

Years ago when I became an assistant scuba instructor, I was able to provide huge value to new students despite my having far less diving experience than all of the other instructors. Simply being able to help them out when I could and knowing the process meant a great deal. On the plus side I learned a lot from the questions they would ask, such that it made me a much better assistant (and eventually full) instructor.

For the same reason I started teaching AST-BBST, I wanted to learn more about the material. I probably wasn’t the best student (frankly I don’t know how well I did in the classes other than passing it) but I loved the information so much I wanted to get more exposure to it. Over time as I interacted with hundreds of students and dozens of instructors I feel like I’ve become exposed to more ideas and absorbed more information than I would have without it. I’m no expert but I’m probably better than most.

All of this is to say the Association for Software Testing is always looking for instructors to help teach classes. The class itself is free (once you’ve successfully completed one AST-BBST class you qualify) but the lessons you learn are incredibly valuable.

How to Write a Good Bug Report, use RIMGEN

RIMGEN is an acronym and mnemonic (or memory aid) to help us remember the elements of a good bug report. It can be used to help anyone write a better bug report or review existing bug reports looking to make improvements. 

In general my preference with reporting bugs is to:

  1. Work with a developer to get them fixed, either by mentioning it passing or pairing with them. By far the most effective way to get bugs fixed.
  2. Fix it myself, although most of these are very simple bugs.
  3. Write a bug report. These are good instructions for that process.

How to Write a Good Bug Report

Writing a bug report seems simple on the surface. People in many roles (programmers, testers, internal and external customers, analysts) write them without much, if any, practice. Although we’ve all seen those bug reports come in without much detail or context and guess what? They don’t get fixed. Just because something is easy to do doesn’t mean it is easy to do well.

We’re writing a good bug report to get a problem fixed. Programmers are busy people, with lots of things going on and plenty of demands on their time. They don’t want to look at a bug report that doesn’t make sense, requires too much work on their part to understand or is offensive. Let’s be honest, if a bug is fairly obvious a programmer isn’t likely to need much information, motivation or influence to fix it. They might not even require a bug report to be written (mention it in passing or through slack). Those aren’t the bugs we are interested in advocating for. We’re advocating for the bugs or problems we understand to be important (for some reason) but might not be easily understood as such.

How do we advocate for bugs to be fixed? We develop reports that clearly communicate the problem, reflect the problem in a harsh but honest way so that some stakeholder realizes the risk and wants to them fixed. Just reproducing the problem might not always be enough. We want to make programmers want to fix the bug by influencing their decision, we’re selling them on fixing the bugs. To “sell” someone on fixing the bugs we need to write a good report.

To help us understand the elements of a good report we can use the heuristic RIMGEN.

RIMGEN:

R – Replicate it

When writing a bug report we should include enough information to get the bug fixed including the steps or conditions necessary to replicate the problem on another machine. The lecture recommends using at least 2 different machines – a powerhouse computer and another less powerful / resource constrained machine. (Virtual environments might work as well.)

I – Isolate it

After we’ve replicated the problem, we want to identify the shortest number of critical steps necessary to replicate the problem by eliminating some of the variables. Clarity is the goal.

We also want to make sure to write up only one problem per report so we don’t get into a situation where one part of the bug is fixed and the other remains open. Example: OpenOffice Impress crashes after importing a file, but our reproduction is 25 steps long. We do some more testing that helps us eliminate some variables until we’re able to get the problem replicated in 6 steps.

M – Maximize it

Maximizing the problem means doing some follow up testing to see if we can find a more serious problem than the one we were originally going to report. Follow up testing can consist of varying our behavior, our environment, our inputs or even our system configuration. The more serious the problem the more likely (or easier) it will be to convince someone to fix it. Example: We find a problem in OO Impress where it won’t import old PowerPoint files; the application just won’t respond. After some follow up testing we find Impress actually crashes if we use an old PowerPoint file over a certain size.

Maximizing a bug (or follow up activities as it’s also known) is one of the more interesting and time consuming aspects of writing a bug report. There’s quite a bit of detail just in the four categories of variables: vary our behavior, vary our environment, vary our data and vary our system configuration and for those interested Markus Gartner goes a little more in-depth about each in this post.

G – Generalize it

Generalizing the bug means seeing if the problem affects more people in more ways than we originally thought. If the bug was a corner case maybe we can un-corner it, show that it occurs under less extreme conditions. (replicating on a broader range of systems?) Example: Same problem as above in OO Impress where it crashes when we try to import old PowerPoint files over a certain size. We vary the data (input) and find out Impress actually crashes when we try to import any file type over a certain size. Now the problem looks really bad and affects more users than just older PowerPoint users.

E – Externalize it

Switch the focus of the report from the program failing to the stakeholders who will be affected by it and show how it will affect them. (Historical review?) Example: OO Impress crashing when importing any file type of a certain size will stop all users from importing comparable application files (like from PowerPoint). This could prevent new users from downloading and installing the application and even turn off existing users who rely on the compatibility.

N – Neutral Tone

Make sure the bug report is easy to understand and that the tone of the report is neutral so as not to offend anyone reading it (perhaps the developer who created the bug) and/or discredit your reputation as the bug reporter (at least in the eye of the reader).

BUG TITLES AND SUMMARIES

To help us communicate (and “sell”) the problem clearly in bug titles and summaries we can use the “Fails, When Protocol”. “Fails, When” says describe the failure first and then when that failure happens second.

For example in Bug Advocacy we look at two (apparently) well known problems in Microsoft Paint 95 (I think that’s the version) related to cutting and pasting from a zoomed section. When I wrote up my problem summary (before understanding the protocol) it came out as: “Cutting (Ctrl-X) a selected area while zoomed causes inconsistent behaviors.” The problem with this summary was it was too vague, contained minor details about the problem and although I spotted two bugs I tried to combine the two summaries into one. If I had used the “Fails, When” protocol a more appropriate set of bug titles might have been: “Cut doesn’t cut (freehand, zoom)” or “Cuts the wrong area (freehand, zoom, resize window)”.

To help us avoid being vague when summarizing and communicating a problem it’s important to consider how much insight the title or summary gives you into the nature, severity, or scope of the problem; does it over emphasize minor details (especially for the bug title)?

A Few Other Things

  • Looking for something a little more portable? Checkout my Bug Reporting Guidelines document.
  • This is an update of my previous post “How to Write a Good Bug Report (and be a Bug Advocate)” with some slight alterations.
  • Design vs coding bugs are slightly different and have a different emphasis using RIMGEN. I hope to address this later in a separate article.
  • Advocating for a bug to be fixed is not an invitation to be sleazy or aggressive in your reporting. We are trying to influence a decision based on our research and understanding, not trying to ruin our credibility by pushing something the business has no desire to fix.
  • RIMGEN is the updated version of RIMGEA

mid-January Updates

Some random thoughts as I sit here at mid-January of the new year:

  • I’ve been reading Walter Isaacson’s newest book Leonardo da Vinci and it’s a fascinating look at how mastery in one discipline or craft such as painting can evolve and become better based on studying other disciplines.
    • Based on Isaacson’s own research including Leonardo’s own notebooks, we are presented with the breath and depth of Leonardo’s self teachings. For example Leonardo often considered himself a scientist, engineer, and weapons designer before a painter. He studied birds and flight, motion, water movement and also dissected animals and human cadavers to learn about muscle movement and skeletal structure. All to learn more about the world around him.
    • All of this cross discipline research influenced and improved his art. I highly recommend the book!
  • I have a Now page, be sure to check it out!
  • Late in December I posted on TestingConferences.org the 2016 & 2017 Conference videos that are free to watch. It was interesting reviewing all of those past conferences and then surveying which ones posted public videos. In the /past list we have a variable called “Event Videos” where we post these. For easy reference:
  • The State of the Testing Survey is now available. There’s a lot of room for interpretation of the results (and questions) but this is by far the best survey of the testing industry. I always fill this out and I hope you will to!
  • The first webinar I’m hosting for AST is coming up. Join us if you haven’t already!

10 Years In

For 10 years (one whole decade) I’ve been employed in a few different software testing positions with a few different titles. It’s been a fun and challenging road. I’ve navigated large companies filled with good people and backward practices to small companies where modern practices are encouraged and my work has to stand on it’s own merit. It’s been a road of struggling to self-educate others and myself on the topic while also moving towards a more empirical model of testing.

Today

I consider myself a reasonably experienced software tester with an interest in the human/machine balance of software development. Ever since Malcolm Gladwell popularized 1 the concept of “10,000-hours of deliberate practice for mastery of a subject”, that mark has become a subject of online discussion and a metric to peg skill levels at, even if it’s usefulness isn’t proven2. Out of curiosity, where does that leave me?

Back of the envelope calculations might go like:

  • I’ve been employed in testing for 10 years. With workdays being 8 hours and 6 of those considered productive, that’s 30 hours a week for 52 weeks a year for 10 years or 15,600 hours. Except, depending on the company, maybe only three to four hours per day was spent on testing, practicing, reporting and hopefully getting better.
    • As for hours of deliberate practice during work, I might be around the 8,000 hour mark.
  • Can’t forget to subtract out sick and vacation days.
  • Need to add in the hours outside of work where I’ve consulted, studied, taken and taught classes on testing, attended conferences and workshops and generally been active in the community
    • That should more than balance out those vacation and sick days.

I might be right at or below the 10,000-hour level of deliberate practice. Without records of my time, days and projects, it’s too difficult to get a realistic number. It’s even harder to see what the advantage or usefulness of compiling such numbers could be, except for fun (which is why I just did it).

Looking Back

When I started this journey I wasn’t sure what I wanted to do. University had come and gone without providing (or without me taking advantage of) many opportunities to learn what I liked or didn’t like. One thing I didn’t like at the time was programming. It wasn’t necessarily hard, but the value was too abstract and the examples didn’t appeal to me. After I graduated, I got a testing job by emphasizing my ability and like of documenting things (writing). Emphasizing documentation in any interview, besides technical writing, suggests I had no idea what I was doing. Looking back on those days, it seems like many of the people I was working with probably had no idea what testing was either (or if they did they decided not share or provide any feedback).

I can’t recall how many interviews I went to over the past decade where there was no testing exercise. As in no demonstration of skills necessary – just come in and try to learn about the system. We hope you do well. Maybe that’s why it took me some time to figure out if I wanted to stay in the field?

At one point I thought project management would be something I’d enjoy more than testing. It seemed more salient. After all I’d been a project leader in college on several projects and although it was stressful, it was rewarding. Eventually I got a project management certificate (nano-degree) in my spare time and came very close to taking the Project Management Professional (PMP) exam. At least until I realized the PMP, although recognizable, really didn’t mean much. In fact during the PMP training, the experienced project managers constantly reiterated there was a difference between what the PMP said to do and what you would do in real life. Many people seemed to be at that training because their employers were convinced the PMP was useful.

Happy that I had first-hand knowledge of the uselessness of the PMP (to me), I went in another direction and started learning more about software testing. I started reading books and attending conferences. The more I learned, the deeper and wider the rabbit whole became. Looking back I’m glad things turned out the way they did.

Looking Forward

It’s a little hard to think of the next decade because it’s so far away and the world is constantly changing. Here’s what I know as of this posting. I’ve been writing (blogging) about software development and testing since 2009, and writing in general since 2008 (although not consistently). That’s slowly changing and I foresee writing on a regular basis. I’ve been co-instructing BBST classes for more than a year now and I’m thinking about stepping up my role in AST and BBST.

In the near future I see more mentoring, collaborating with peers, starting a meetup and/or a workshop and hopefully producing a wider variety of public work. I’m working harder at applying programmatic solutions to problems and constantly on the look-out for examples. There’s still a lot to do.

I haven’t yet:

  • Given any talks,
  • Published any conference papers,
  • Attended any meet ups,

At 10 years in, I’m working harder than ever, on a better path and more determined. Here’s to the next decade wherever that may take me!

The idea of a Professional Tester

As rough as traveling can be, one benefit is dedicated time to catch up on reading. I finally got around to a post from Uncle Bob on “Sapient Testing: The “Professionalism” meme” and it captured something I’ve been thinking about for a some time: the label of professional(ism).

I’ve been using the “professional” term online to call attention to a potential difference between myself and everyone else: that I take my work seriously. Being a professional isn’t just about being paid to do something – there are many people paid to do jobs they could care less about (airports are a constant reminder of this); I mean professional in the sense of caring about the quality of my work, my skills and my ability to stay relevant.

After reading what Uncle Bob wrote after he attended a keynote by James Bach on the topic of professionalism, I think this says it better:

A professional tester does not blindly follow a test plan. A professional tester does not simply write test plans that reflect the stated requirements. Rather a professional tester takes responsibility for interpreting the requirements with intelligence. He tests, not only the system, but also (and more importantly) the assumptions of the programmers, and specifiers.

Uncle Bob goes on to say:

I like this view. I like it a lot. I like the fact that testers are seeking professionalism in the same way that developer are. I like the fact that testing is becoming a craft, and that people like James are passionate about that craft. There may yet be hope for our industry!

I like this view as well; I like it a lot.

Shaping Your Identity as a Tester

On Thursday, June 25, 2015 I presented my first webinar called Shaping Your Identity as a Tester that was based on an earlier article I wrote called Blogging for your Career.

uTest recorded it and made it part of their uTest University series, you can check it out here. I’ve also embedded the video below for easy viewing. The slides are pretty simple. My part lasts less than 30 minutes, the rest of the time is spent answering questions fielded from the 75 or so participants who joined. Check it out:

If you joined the webinar and your question wasn’t answered or if you have any questions after watching it here, feel free to leave me a comment (or contact information) or connect with me (on the right) and I’d be happy to respond!

The Anti-Library

The writer Umberto Eco belongs to that small class of scholars who are encyclopedic, insightful, and nondull. He is the owner of a large personal library (containing thirty thousand books), and separates visitors into two categories: those who react with “Wow! Signore, professore dottore Eco, what a library you have! How many of these books have you read?” and the others – a very small minority – who get the point that a private library is not an ego-boosting appendage but a research tool. Read books are far less valuable than unread ones. The library should contain as much of what you don’t know as your financial means, mortgage rates and the currently tight real-estate market allows you to put there. You will accumulate more knowledge and more books as you grow older, and the growing number of unread books on the shelves will look at you menacingly. Indeed, the more you know, the larger the rows of unread books. Let us call this collection of unread books an antilibrary.

Nassim Nicholas Taleb in the Black Swan

A Collection of Menacing Books

The more I attend conferences, interact with others and read blogs in the various software communities I take interest in (testing, entrepreneurship, technology, etc.) the greater the number of books I find I’d like to consume. It seems almost automatic these days to ask for books or references someone has found useful to solve problems or gain a wider (or deeper) perspective on a subject.

Yet a collection of books on a list or sitting on in Dropbox isn’t nearly as menacing and doesn’t call to me the same way as having those books sitting on my shelf where I have to watch them age.

No Clutter
I try to find balance between only owning those things I can and do use regularly and a more consumerist desire where I need to have lots of options. I’m more on the minimalist side where I want no clutter (of anything) so I constantly edit what I own. Part of this comes from wanting to be practical and trying to understand trade-offs between buying something now or something better later. Part comes from the fear of getting “behind” or having too many choices.

This manifests in many ways including trying not to accumulate too many books.

A Research Tool

When I first toured Cem Kaner’s personal library I fell into the first category of people from the quote above. I think I said, “Wow what a library”, as the size of his personal collection was more immense than anything I’d seen. Now it wasn’t comparable in size to Umberto Eco’s but it was impressive enough that I felt the need to tell him so. My parents have book shelves filled with books they read twenty years ago but Cem had bookshelves full of books he hadn’t read and those he kept around for reference. After my “impressive” comment, Cem made it clear the point of his private library was not built to impress but as a research tool.

Keeping materials around for research or reference makes sense. Some you might keep around because you feel a special attachment to or because they gain in value but most books don’t. They aren’t trophies either.

The Anti-Library

The anti-library (or antilibrary) seems like the logical compromise between acquisition and usefulness. Get rid of those books you’ve read and aren’t likely to reference again and keep the collection of books you haven’t. Don’t make it your library, make it your anti-library.

The Domain Testing Workbook is available

Cem Kaner, Sowmya Padmanabhan and Doug Hoffman have a new book called The Domain Testing Workbook. I’d highly recommend picking up a copy or at least adding it to your reading list! This book is not just a deep dive into one test technique but it represents a collective thinking about what software testing is today.

Domain Testing

BBST Test Design was my formal introduction to Domain Testing aka boundary and equivalence class analysis. Domain Testing is often cited as the most popular (or one of the most popular) test techniques in use today. For its part the Test Design course spends a whole week, a full lecture series and at least one assignment introducing and practicing this technique. If you’d like an introduction I recommend the first lecture from the fifth week in the Test Design series in which Cem introduces Domain Testing:

(For more information see the Testing Education Website or YouTube.)

I got the chance to do some reviewing of the workbook and enjoyed how it elaborated on the material I was learning Test Design. Yet it went further by offering layers of detailed examples to work through and is set up to allow the reader to skip around to whatever section or chapter they find interesting.

In an email, Cem described the book:

My impression is that no author has explored their full process for using domain testing. They stop after describing what they see as the most important points. So everyone describes a different part of the elephant and we see a collection of incomplete and often contradictory analyses. Sowmya, Doug and I are trying to describe the whole elephant. (We aren’t describing The One True Elephant of Domain Testing, just the one WE ride when we hunt domain bugs.) That makes it possible for us to solve many types of problems using one coherent mental structure.

Cem Kaner

I’m excited for this book to be turned into a class on domain testing ( and hopefully open sourced) with the rest of BBST. In the meantime pick up the book from Amazon and let me know what you think!

Anyone Can Test, right?

In this video from StarEast Rob Sabourin talks about his experience with concept of “anyone can test”. This actually gives me an idea for a challenge:

Before watching, make a list or jot down some notes that try to describe the potential benefits and problems with assigning any particular person to test your product or service. Finally what the video and do a comparison. How many things did you come up with vs. Rob. Then formulate your own response to can anyone test?

Anyone can test, right?