Conference Idea – A Bug Advocacy Boot Camp?

If someone asked you to speak at or submit a proposal to speak at a conference do you know what you’d say?

I’d say no thanks for now, but I do have an idea for a tutorial. (This is a not-so-well thought out brainstorm and not a proposal.)

Anyone who’s read my blog of late knows I’m a fan of the Black Box Software Testing courses and anyone who has taken a course knows there’s more material cover than the typical 4 week class can get to. It’s very good material but it’s overwhelming at times. One of the classes, Bug Advocacy, has an immediate real world impact for practitioners. If the class could be sliced up in a way that helped participants understand:

  • Basic concepts (quality, bug workflow) 
  • The value of writing good bug reports for the bug writers (testers, programmers, etc.) 
    • For the Bug Reporters – testers primary work product and they put our credibility on the line 
    • For the organization – Well written bug reports can have an impact on quality and the development lifecycle 
    • Motivating the Bug Fixer
  • How to Write a Good Bug Report 
  • This boot camp is not a supplement to the full Bug Advocacy course

I think it could be a valuable Full Day Tutorial. Not as valuable as taking the full class but still valuable for a professional conference.

The challenges are numerous. This is typically a four week course with six recorded lessons, hundreds of slides, a final written exam and a few assignments that include requiring students to analyze a product and write persuasive, well written bug reports. The class also requires students to join the Open Office project, select a few poorly written bug reports, analyze them and submit re-writes that are better. There’s also a lot of time for peer-review and comparisons of well written and not so well written bugs.

Perhaps by focusing more on the Learning Objectives and less on content with some staged / preplanned simple and complex assignments would work?

This idea is interesting for a number of reasons:

  • It would further my understanding of the material and how to teach it (instructional design) which is part of the reason I became a BBST Instructor (I’d like to one day teach the material on my own). 
  • Bug Advocacy has enough practical material that it would be compelling to students who may not be aware of the BBST classes.
    • Further raise awareness for BBST classes and those who teach it (AST & KFA) 
  • Get more people involved in the open courseware community and potentially lead to better material.

I’d want a number of trial runs with this new instructional design before I applied to a conference. Perhaps after a few classes in my new role as an AST-certified BBST Instructor and then by some willing participants at a full day meet up I might have a better idea of the viability of a boot camped / introduction to Bug Advocacy. Maybe there would be a way to include the eventual Bug Advocacy Workbook in the class or as a way for the students to follow up. (Note the Foundations Workbook has just been released and I have no idea when the Bug Advocacy one will be out.)

Any ideas or advice?

(Thanks to Becky Fiedler who suggested Bug Advocacy as the BBST class most appropriate for a full day conference during a discussion we had during WTST / BBST Domain Testing last month.)

I’m an AST BBST Instructor

That’s right, I completed the trifecta of BBST classes and decided to continue on to being an Instructor. I finished the class in November but just recently got the proof:

Boom! Now comes the hard work – working as an assistant through enough classes until I learn the system and work my way up to a lead instructor role for AST.

Part of my desire for taking the BBST Instructor class is to become more familiar with material taught in the BBST classes, get exposed to new student’s ideas (that will hopefully broaden and challenge my understanding) and learn how to provide important and useful feedback (and help others do the same thing).

Eventually I hope to teach the BBST materials to people I work with and help expose others to the material outside of AST. This was the first step.

State of Testing Survey 2013

Lalit Bhamare a co-founder and editor of Tea Time with Testers magazine contacted me to share a project he and Joel Montvelisky came up with – a State of Testing survey. Lalit said it best when he described why they wanted to undertake this survey:

Some weeks ago my friend, Joel Montvelisky was looking for information to write a post about the advances in the testing world in the last 5-10 years and he realized that there is no centralized set of information that provides visibility into what is happening and what are the trends in the world of testing today. In principle he was looking for something similar to the State of Agile survey that he reviews each year when it goes out, and he was not able to find something that provided such information.

Lalit Bhamare

So Joel contacted me and we decided to turn this into a project i.e. to conduct a State of Testing Survey that will provide a snapshot of our testing reality and help us to capture some of the trends as they shift year by year.

Lalit Bhamare

They want to understand a few things about the testing community (click to see a bigger image):

I think this is a great idea so I signed up to be notified when the survey goes live. I encourage you to do the same by going to http://qablog.practitest.com/state-of-testing/. Oh and hurry it will only be available for 10 days, starting towards the end of this week!

See you at WTST 2014

The Workshop on Teaching Software Testing, abbreviated as WTST and I think pronounced “what’s it”, is coming up at the end of January 2014 in Melbourne, Florida near the Florida Institute of Technology campus. (You maybe have seen a few of Cem Kaner’s posts.) Just recently my application was approved and I’m excited to be attending as a non-presenting participant.WTST is a LAWST-style workshop which means unlike regular conferences it’s limited to small group of active participants, some who present and everyone is expected to engage in discussions by asking question, providing their experiences and even thoughtfully arguing with the presenter. Discussion is the entire point of the workshop.

This year WTST is focused on designing and teaching advanced courses in software testing, something I’ve had an interest in for a few years now. Teaching software testing was the driving force to me becoming an Instructor for the Association for Software Testing’s BBST classes (which I completed just recently).

This year the workshop will be broken into two sections: The first is the typical weekend-long WTST workshop and the second is the Domain Testing Workshop held for five days immediately after the traditional WTST. I’m excited to be part of a pilot course built around The Domain Testing Workbook (see my previous post for more info) and I imagine it will be a fun exercise in understanding domain testing a little better.

Who else is going? Hopefully I’ll see you there. =)

For more information see http://wtst.org.

The Domain Testing Workbook is available

Cem Kaner, Sowmya Padmanabhan and Doug Hoffman have a new book called The Domain Testing Workbook. I’d highly recommend picking up a copy or at least adding it to your reading list! This book is not just a deep dive into one test technique but it represents a collective thinking about what software testing is today.

Domain Testing

BBST Test Design was my formal introduction to Domain Testing aka boundary and equivalence class analysis. Domain Testing is often cited as the most popular (or one of the most popular) test techniques in use today. For its part the Test Design course spends a whole week, a full lecture series and at least one assignment introducing and practicing this technique. If you’d like an introduction I recommend the first lecture from the fifth week in the Test Design series in which Cem introduces Domain Testing:

(For more information see the Testing Education Website or YouTube.)

I got the chance to do some reviewing of the workbook and enjoyed how it elaborated on the material I was learning Test Design. Yet it went further by offering layers of detailed examples to work through and is set up to allow the reader to skip around to whatever section or chapter they find interesting.

In an email, Cem described the book:

My impression is that no author has explored their full process for using domain testing. They stop after describing what they see as the most important points. So everyone describes a different part of the elephant and we see a collection of incomplete and often contradictory analyses. Sowmya, Doug and I are trying to describe the whole elephant. (We aren’t describing The One True Elephant of Domain Testing, just the one WE ride when we hunt domain bugs.) That makes it possible for us to solve many types of problems using one coherent mental structure.

Cem Kaner

I’m excited for this book to be turned into a class on domain testing ( and hopefully open sourced) with the rest of BBST. In the meantime pick up the book from Amazon and let me know what you think!

Lee Kuan Yew: The Grand Master’s insights on China, the U.S. and the World

Lee Kuan Yew has been the prime minister of Singapore (an island nation or city-state) for about 50 years and is credited with taking them from a developing country into one of the world’s largest economies as measured by Purchasing Power Parity. Singapore has an interesting story, transforming itself in just a few decades from almost irrelevant to a top nation economically.

I don’t recall how I first heard of Lee Kuan Yew (LKY) or how he became a topic of conversation but his accomplishments as a political leader seemed interesting enough that I wanted to learn more so I bought The Grand Master book. As I was reading the book I also saw his interview with Charlie Rose. At an age (90) he is slow and quiet to answer things but it’s good watch if you want to learn more about him (and have an hour). Some of the things they talk about are also covered in the book:

The book is broken into chapters that discuss the future of China, the U.S., U.S.-China relations, India, Islamic Extremism, globalization, democracy and the way LKY thinks. Each chapter presents a dozen or so questions and the answers are provided through a selection of interviews picked by the authors. This makes the book a compilation of very interesting and diverse interviews about the above topics but it also means some of the selected replies are duplicates, which can, at times, be a little confusing.

Some of the highlighted points include:

  1. It’s China’s intention to be the greatest power in the world and its neighboring countries have already taken this into account. They’ve repositioned themselves because they know there will be consequences if they try to stop China. China can simply deny access to its 1.3 billion people whose incomes and purchasing power are increasing.
  2. China is getting more aggressive as its position in the world increases and it understands it can deny access to its markets.
  3. China has a lot of handicaps going forward like the abuse of the rule of law (the Chinese still behave like they are run by an emperor); a huge country with little emperors running around controlling local areas; cultural habits that limit creativity, imagination and expression; a language that shapes thinking around epigrams and 4,000 years of text that suggests everything worthwhile has been said; a language that is difficult for foreigners to learn; and the inability to attract and assimilate talent from other societies around the world.
  4. China is not going to become a liberal democracy; it would collapse if it did. There won’t be a revolution for democracy either, just look at Tiananmen Square and the impact that’s had – very little.
  5. The U.S. is going through a rough patch with its debt and deficits but America will not fall to second-rate status. Historically the U.S. has demonstrated a great capacity for renewal and revival thanks to a wide range of imagination and pragmatics; a diverse population that competes in investing, embracing new ideas, taking risks; a society that attracts talent from around the world and assimilates them comfortably; and a language that is an open system that is the language of the leaders in science, technology, business, etc.
  6. In the US Presidents don’t get reelected if they give a hard dose of medicine to their people so there is a tendency to procrastinate, to postpone unpopular policies in order to win elections. This is how we get budget deficits, debt, and high unemployment carried over from one administration to the next.
  7. The US’s approach towards China with human rights groups, threatening the loss of most-favored-nation status, etc. ignore the differences of culture, values and history and hurt China-U.S relations. Less sensitivity and more understanding of the cultural realities of China can make the relationship better.
  8. India has wasted decades in state planning and controls that have bogged it down in bureaucracy and corruption. A decentralized system would allow more centers like Bangalore and Bombay to grow. The caste system has been the enemy of meritocracy. India is a nation of unfulfilled greatness.
  9. There are limitations in the Indian constitutional system and political system that prevent it from going at a high speed. Whatever the political leadership wants to do it has to go through a very complex system.
  10. Islam has not been a problem; however contemporary radical Islam or Islamism is a problem.
  11. The Russian population is declining. It is not clear why but alcoholism plays a role; so do pessimism, a declining fertility rate, and a declining life expectancy.
  12. There is no historical precedent on how to maintain peace and stability and to ensure cooperation in a world of 160 nation-states. And the age of instant communications and swift transportation, with technology growing exponentially makes this problem very complex. In one interdependent, interrelated world, the decline in the relative dominance of the leaders of the two blocs increases the likelihood of a multipolar world and with it the difficulties of multilateral cooperation.
  13. Westerners (The US) have abandoned an ethical basis for society, believing that all problems are solvable by a good government… In the West, especially after WW2, the government came to be seen as so successful that it could fulfill all the obligations that in less modern societies are fulfilled by the family… In the East, we start with self-reliance. In the West today, it is the opposite. The government says give me a popular mandate and I will solve all society’s problems.

There’s a lot more detail is this book of nearly 200 pages, for example LKY goes more into the U.S.’s problems, like the war on drugs and the US’s tolerance for illegal immigrants but is optimistic in our ability to solve problems when we finally come together.

lky

I would definitely recommend this book to someone wanting to learn more about Lee Kuan Yew’s views of the world and/or anyone interested in International Politics / Relations. Given the international views we (I) have as (an) American’s it’s refreshing to read someone else’s views on the world and how the US fits in. That’s not to say LKY is a know-it-all, or doesn’t make mistakes but his leadership is unrelenting.

The Gang and Us

Last night on It’s Always Sunny in Philadelphia the episode we were on set to make aired called The Gang Tries Desperately to Win an Award. It’s on Amazon, iTunes and possibly other places but I bought it and cut out the little section we are in.You’ll find us in the background at roughly 0:04 giving a high-five to the bartender and at 1:11 and 1:35 for our close ups. Ah our 4 seconds of fame; well worth the 10 hours on set. lol.

It’s Always Sunny in Philadelphia Season 12, Episode 3. Enjoy:

 

How to Write a Good Bug Report (and be a Bug Advocate)

For the previous BBST classes I made the mistake of not writing about what I learned and the value the classes provided me. To an organization who might consider sponsoring someone (a programmer, analyst…)  or to a fellow tester who is looking for a step in the right direction I have no story to tell, until now.

For Software Testers bug reports are one of our most visible work products – something tangible that comes from our efforts of investigating the system. How well testers write a report can effect how we are perceived (to those who read them) which in turn effects our credibility and influence among those same people (especially programmers). Specifically for development and testing groups, bug reports are important communication mediums (think about the amount of time you spend reading and writing them) and as such an emphasis on understanding bug reports and improving writing skills is important. This is why I took the  Bug Advocacy course through AST and here’s what I learned.

How to Write a Good Bug Report

Writing a bug report seems simple. People in many roles (programmers, testers, internal and external customers, analysts) write them without much, if any, experience but just because something is easy to do doesn’t mean its easy to do well.

So what’s the point of writing a bug report? To get the bug or problem fixed! That’s why the lead author of the material, Cem Kaner, calls it Bug Advocacy – we are advocating for a fix. Programmers are busy people, with lots of things going on and plenty of demands on their time. They don’t want to look at a bug report that doesn’t make sense, requires too much work on their part to understand or is offensive. Let’s be honest, if a bug is fairly obvious a programmer isn’t likely to need much information, motivation or influence to fix it. They might not even require a bug report to be written. Those aren’t the bugs we are interested in advocating for. We’re advocating for the bugs or problems we understand to be important (for some reason) but might not be easily understood as such.

How do we advocate for bugs to be fixed? We develop reports that clearly communicate the problem, reflect the problem in a harsh but honest way so that some stakeholder realizes the risk and wants to them fixed. Just reproducing the problem might not always be enough. We want to make programmers want to fix the bug by influencing their decision, we’re selling them on fixing the bugs.

To “sell” someone on fixing the bugs we need to write a good report. To help us understand the elements of a good report we can use the heuristic RIMGEA:

R – Replicate it. When writing a bug report we should include enough information to get the bug fixed including the steps or conditions necessary to replicate the problem on another machine. The lecture recommends using at least 2 different machines – a powerhouse computer and another less powerful / resource constrained machine. (Virtual environments might work as well.)
I – Isolate it. After we’ve replicated the problem, we want to identify the shortest number of critical steps necessary to replicate the problem by eliminating some of the variables. Clarity is the goal. We also want to make sure to write up only one problem per report so we don’t get into a situation where one part of the bug is fixed and the other remains open. Example: OpenOffice Impress crashes after importing a file, but our reproduction is 25 steps long. We do some more testing that helps us eliminate some variables until we’re able to get the problem replicated in 6 steps.
M – Maximize it. Maximizing the problem means doing some follow up testing to see if we can find a more serious problem than the one we were originally going to report. Follow up testing can consist of varying our behavior, our environment, our inputs or even our system configuration. The more serious the problem the more likely (or easier) it will be to convince someone to fix it. Example: We find a problem in OO Impress where it won’t import old PowerPoint files; the application just won’t respond. After some follow up testing we find Impress actually crashes if we use an old PowerPoint file over a certain size.
Maximizing a bug (or follow up activities as it’s also known) is one of the more interesting and time consuming aspects of writing a bug report. There’s quite a bit of detail just in the four categories of variables: vary our behavior, vary our environment, vary our data and vary our system configuration and for those interested Markus Gartner goes a little more in-depth about each in this post.
G – Generalize it. Generalizing the bug means seeing if the problem affects more people in more ways than we originally thought. If the bug was a corner case maybe we can un-corner it, show that it occurs under less extreme conditions. (replicating on a broader range of systems?) Example: Same problem as above in OO Impress where it crashes when we try to import old PowerPoint files over a certain size. We vary the data (input) and find out Impress actually crashes when we try to import any file type over a certain size. Now the problem looks really bad and affects more users than just older PowerPoint users.
E – Externalize it. Switch the focus of the report from the program failing to the stakeholders who will be affected by it and show how it will affect them. (Historical review?) Example: OO Impress crashing when importing any file type of a certain size will stop all users from importing comparable application files (like from PowerPoint). This could prevent new users from downloading and installing the application and even turn off existing users who rely on the compatibility.
A – And say it clearly and dispassionately. Make sure the bug report is easy to understand and that the tone of the report is neutral so as not to offend anyone reading it (perhaps the developer who created the bug) and/or discredit your reputation as the bug reporter (at least in the eye of the reader).

To help us communicate (and “sell”) the problem clearly in bug titles and summaries we can use the Fails, When Protocol which says describe the failure first and then when that failure happens second. For example in the Bug Advocacy class we looked at two (apparently) well known problems in Microsoft Paint 95 (I think that’s the version) related to cutting and pasting from a zoomed section. When I wrote up my problem summary (before understanding the protocol) it came out as: “Cutting (Ctrl-X) a selected area while zoomed causes inconsistent behaviors.” The problem with this summary was it was too vague, contained minor details about the problem and although I spotted two bugs I tried to combine the two summaries into one.

If I had used the protocol a more appropriate set of bug titles might have been: “Cut doesn’t cut (freehand, zoom)” or “Cuts the wrong area (freehand, zoom, resize window)”.  To help us avoid being vague when summarizing and communicating a problem it’s important to consider how much insight the title or summary gives you into the nature, severity, or scope of the problem; does it over emphasize minor details (especially for the bug title)?

How to be a Bug Advocate

We want to make programmers, managers and other stakeholders want to fix the bugs and we can do that by influencing their decision. Writing clear reports can lead to that happening but it’s important to be aware of other things that can influence our ability to become effective Bug Advocates (and effect our ability to write those reports):

  • There are lots of definitions of a software error (so make sure you understand your organizations or teams).
  • There are lots of definitions of quality (so make sure you understand your organizations or teams).
  • Report readers (developer, testers, customers) carry biases with them. Try to minimize the effect these have on them to get the problems fixed.
  • Keep in mind there are lots of reasons why a bug might not get fixed:
    • Time constraints of the people involved
    • Cost of fixing is too prohibitive
    • Trade off between the cost of fixing and adding a feature people like (loss in perceived value due to the problem, an increase in value by adding a new feature customers want, opportunity cost, etc.)
    • Credibility of the bug writer is low in the readers eyes (manager, programmer, etc.)
    • Problem is a corner case (not likely to affect a lot of users)

The value of Bug Advocacy

The value for the bug writer (tester, programmer, analyst, etc) is pretty simple:

  • Experience writing clearer and more effective bug reports
    • Better bug reports can increase the tester’s credibility in the reader’s eyes (programmer)
  • Experience evaluating other people’s bug reports and providing constructive feedback for them to improve
  • Greater understanding of the details / elements involved in the communication medium of bug reports (think biases)
  • Contribute time and effort to an Open Source project (think giving back to the community)
  • Public references to your Open Source work (think for your portfolio)
  • Better understanding in the trade offs between fixing and not fixing problems

The value of the class for an organization thinking of sponsoring an individual:

  • Better written, evaluated and understood bug reports which can lead to a greater efficiency (time saved reading) and understanding of new reports.
  • Increased desire in going back through existing reports, evaluating them and improving them (increasing the likelihood they get addressed).
  • Increased credibility of bug report writers (might help identify and decrease existing biases).
  • Better understanding of organization bias
  • Better understanding of organization needs

I’d like to leave you with one of my favorite quotes from the Bug Advocacy course (courtesy of Cem Kaner):

The best tester isn’t the one who finds the most bugs or embarrasses the most programmers. The best tester is the one who gets the right bugs fixed.

To do that you must have credibility, understand the scope of quality for your organization, have experience writing effective bug reports and in general be a bug advocate.

Installing Ruby and Watir on Windows 7

I recently started playing around with Web Application Testing in Ruby or Watir for short. As is recommended, I downloaded the Homebrewer’s Guide to Watir by Zeljko Filipin and started trying to install the latest versions of the RubyInstaller for Windows (currently 2.0.0) when I ran into a few problems. (I like the idea of using Leanpub to create and modify install documents for open source applications.)

I installed Ruby and Watir on a Windows 7 64 bit machine using the 32 bit versions and everything seems to work fine so far. Here’s how I installed everything using the RubyInstaller and instructions from the above guide (note: some of these instructions will be a duplicate of the guide):

Ruby Installation

  1. Open a command prompt (Run and type cmd) and type ruby -v to see if you have Ruby installed (if you don’t it will say the command isn’t recognized) and the version number.
  2. I have Windows 7 64 bit but I used the 32 bit installers from RubyInstaller Downloads including:
    1. Ruby 2.0.0-p247
    2. DevKit-mingw64-32-4.7.2
  3. Run the Ruby installer and on the optional tasks page select the below items before completing:
    1. Add Ruby executables to your PATH
    2. Associate .rb and .rbw files with this Ruby installation
  4. Open a new command prompt and type ruby -v to see if Ruby installed and the version number.
  5. (Optional) Once installed you can update the RubyGems distributed with the installer by typing gem update --system and watching for the “RubyGems system software updated” to know when its complete.
  6. Move on to the DevKit installation.

DevKit Installation

  1. Run the DevKit installer but change the extraction folder to C:devkit
  2. Open a command prompt (Run and type cmd) and change the folder to C:devkit (use the command cd c:devkit).
  3. Run the command ruby dk.rb init. If this step is successful you’ll see a response like “[INFO] found RubyInstaller v2.0.0 at C:/Ruby200”.
  4. Run the command ruby dk.rb install. If this step is successful you’ll see a response like [INFO] Installing C:/Ruby200/lib/ruby/site_ruby/2.0.0/
    rubygems/defaults/operating_system.rb
    and [INFO] Installing C:/Ruby200/lib/ruby/site_ruby/devkit.rb
  5. Move on to Watir installation.

Watir Installation


This is where I ran into problems with the 2.0.0 version of Watir. Something about the mini_magick version erroring out. To prevent this problem we do:

  1. Run the command (still from the C:devkit command window) gem install mini_magick -v 3.5.0 which works around the version problem. You should get a response like “2 gems installed”
  2. Then run the command gem install watir --no-ri --no-rdoc to install the rest of Watir. You should get a response like “Successfully installed watir-4.0.2-x86-mingw32 and 23 gems installed”.

Check the installation

In the same command window type (after each line of code submit the commands)
  1. irb
  2. require "watir"
  3. browser = Watir::Browser.new
  4. browser.goto "google.com"

If all is setup correctly you should get your default browser to open a new window and then browse to Google. Good luck.

Note: These steps weren’t meant to replace the Homebrewer’s Guide to Watir by Zeljko Filipin instructions but to improve them for the 2.0.0 release. For everything else please refer to the guide.

I’m a Bug Advocate

I do advocate for bugs to be fixed but the title comes from passing the Association for Software Testing‘s (AST) Black Box Software Testing (BBST) Bug Advocacy course. The class officially ended in mid-July and it marks the third and final BBST class for me. Together Foundations, Bug Advocacy and Survey of Test Design make up a semester long class that Cem Kaner teaches at Florida Institute of Technology called Software Testing 1. These lectures and classes are, as far as I know, the only college level software testing courses available to anyone that wants to learn about testing – the material is free and the courses are pretty inexpensive.

As with the prior classes I agreed to have my name listed on AST’s graduates website and for fun my certificate of completion is below:

Despite this being the final BBST class my journey with BBST doesn’t end here. I enrolled in the Instructor class for November at the beginning of the year when I enrolled in Test Design. I figured the best way to become more familiar with the material and reinforce what I do know (challenge what I don’t know) is to try to teach it. Worked for Scuba Diving!