Anyone Can Test, right?

In this video from StarEast Rob Sabourin talks about his experience with concept of “anyone can test”. This actually gives me an idea for a challenge:

Before watching, make a list or jot down some notes that try to describe the potential benefits and problems with assigning any particular person to test your product or service. Finally what the video and do a comparison. How many things did you come up with vs. Rob. Then formulate your own response to can anyone test?

Anyone can test, right?

AST Membership and Learning Goals

It’s official I’m a member of the Association for Software Testing or AST as it’s commonly known.

I’ve been meaning to sign up so I can take the BBST Foundations Course, meet some local (or not so local) context-driven testers, perhaps post on their discussion boards and eventually head to CAST (I’m aware you don’t have to join to go).

I’ve been bouncing around the idea of setting up some type of local tester chapter / meet up place where testers can get together, train with each other, perhaps join in a weekend tester session, learn from each other, etc. The problem is I’m not sure how to go about doing it.

In other news I also signed up for Udacity’s Software Testing (CS258) class. I’m not a programmer and it does require Python programming experience so I’m going to focus on getting up to speed before the class. I’m curious as to what they’ll teach although the syllabus gives some hint. Units include:

  1. Introduction
  2. Domains, Ranges, Oracles, and Kinds of Testing
  3. Code Coverage
  4. Random Testing
  5. Advanced Random Testing
  6. Consequences
  7. Conclusion
The mention of Oracles has me interested and so does the “kinds of testing”. I wonder how this relates to Test Techniques? Based on the site description the class seems geared more towards programmers for example unit 4 and 5 are about automatically generating test cases (Random and Advanced Random Testing). Regardless I convinced one of my co-workers (who’s a programmer) to take it with me; I might be able to learn something about testing from the programmers perspective.
If you read my previous post What Testers Need to Learn some of the areas James tells tester’s to understand include Mathematics and Technology. Udacity has an Intro to statistics and quite a few technology classes that could potentially help testers.

What Testers Need to Learn

Sunday night I attended a live webinar by James Bach entitled “What Testers Need to Learn” that was put on by Tea time with Testers. It seemed like an interesting topic so I joined (it only cost $30).

The webinar got off to a slow start thanks to some technical issues with GoToMeeting but as soon as they were resolved James jumped into his talk: his personal vision of the skills testers need to have based on his many years of experience coaching testers.

James shared his recently updated tester’s syllabus (a free download from his site) and then walked through it explaining some of the areas. The syllabus he shared was actually a part of a specially created slide deck composed of existing materials but arranged for this talk. You can download the slide deck here. If you haven’t seen (or downloaded) the syllabus these are the main areas:

  1. General Systems
  2. Applied Epistemology
  3. Social and Cognitive Science
  4. Mathematics
  5. Testing Folklore
  6. Communication
  7. Technology
  8. Software Process Dynamics
  9. Self-Management

A synopsis (what I remember) of the walk-through:

General Systems theory involves understanding what makes systems complex. It’s a fundamental skill of testing based on how to approach a system, break it down and then understand what’s there. James recommends An Introduction to General Systems Thinking by Weinberg.

Applied Epistemology. Epistemology is the study of how we know what we know. Scientific Thinking helps us understand applied epistemology. Testing is the process of creating experiments and exploring them so understanding how to design experiments is also very critical. Understanding written and un-written requirements requires an understanding of epistemology.

Cognitive Science. The difference between how people should think (a factor in epistemology) and how people do think is cognitive science. As a tester we we need to understand how people’s perception and biases factor into their work. Human factors relate to how people use and misuse systems. Testers are constantly learning so learning theory is huge.

Mathematics. Testers seem to be bad or afraid of mathematics and what you get is a system where people misuse / abuse mathematics. Counting test cases or metrics are often faulted here. People who don’t understand mathematics are too afraid to ask or question assumptions. The number one thing James uses is combinatorics or as he describes it “counting things in combinations”. Graph Theory is also big for identifying different pathways in testing. You don’t need to be an expert in these things but you need to know enough to be comfortable learning more.

Testing Folklore. Folklore dominates the testing field today. Testing folklore are ideas that are widely spread in the industry and yet poorly found in scientific practice / not based on scientific method. Examples include listing testing techniques, poorly thought out definitions or lists of words, testing certification, most things the ISTQB does, certain ideas about what testing is, etc. Communities of testers are found here including the context-driven school that James belongs to. Highly educated testers need to understand this, if you don’t want to be an expert you can ignore it. (Gotta love his attitude.)

Communication is important because testers need to write, make reports, and design documentation for the appropriate audience. Social Legibility involves presenting yourself and your work in a way people think they can understand.

Technology. The more you know about technology the better you’ll be as a tester. For example, programmers who don’t know anything about testing can be good in many respects because their knowledge of technology is great. This seems to be the area most testers understand the most. It’s helpful to know technology but great testers need to know about the other things as well.

Software Process Dynamics is important though not as important as other things. (No mention of how it ranks among its peers.) Software process dynamics is about how projects go wrong, why its good to find certain types of bugs at certain times, etc.

Self-Management. A lot of things are lumped in here because it’s a big deal in testing. This area is entirely non technical and is about being a grown-up: make plans and then do things. Ethical issues, contracts, accounting, record-keeping, being helpful are all lumped in here.

After the overview of the syllabus James answered some participant questions.

The first was asked by a gentleman who worked or was related to Tea time with testers. I think the question was how testers should balance learning with time commitments and how effective someone can be at learning. James’ response was something like:

You have to learn on the job, then work nights and on the weekends to be a great tester. Read a lot and try to experiment with a lot of things. Weekend Testing can help. He and others offer free coaching sessions via Skype. Other options include working with a peer group or other like-minded people, preferably not alone. Try to find somebody to work with and if you come up empty ask James. Also build a step by step portfolio of your work – where a portfolio is a set of your work that you can show to other people that demonstrates what you do as a tester.

There were a few more questions but since this webinar took place from 10:20 PM PST until 11:45 PM PST and those questions didn’t interest me I didn’t pay attention. =)

James mentioned a few of his recommended books, found on page 6 of the slide deck. Like I mentioned above I’ve posted a copy in my Dropbox folder here. All the people James respects as experts read and study books veraciously. They also have a support group that does the same.

The last part of the talk focused on skills he sees missing or needing improvement in the people he coaches – also found in the slide deck. Rapid Technical Self-Education, Test Framing, Test Factoring, etc. are some of the skills he focuses on when coaching. Student Syndromes are common problems he sees.

Lastly James shared and briefly discussed his Exploratory Testing Dynamics paper. Unfortunately at this point I don’t remember what was said about it. I was recording the whole broadcast (which hopefully someone will make available online) but some how it managed to crash and I lost my recording.

My (testing) learning problems are related to asking for help, not within my team or company but with people I don’t know. I work alone the majority of time but really I need someone to work with that can help push me. I think it’s time for a coaching session from James.

1993 World Book definitions for Quality and Testing

According to the 1993 “The World Book Dictionary” the definition for Quality Control is

“[T]he inspection of manufactured products from the raw materials that go into them to their finished form to insure that they meet the standards of quality set by the manufacturer.” (pg. 1703.)

That same dictionary didn’t have any definition for the word quality assurance and had many definitions for the word quality (7 to be exact).

The most relevant definition for Tester was defined as:

“[A] person or thing that tests.” (pg. 2167)

My parents still have their set of 1993 World Book encyclopedia’s which came with a two book set of dictionaries.

Throw someone else in to help QA it faster!

“Throw someone else in to help QA it faster!”

A former boss (or two) of mine

I’ve heard this statement many times in my career but it happened again just recently and it got me thinking. Aside from the poor choice of words, about “QAing” something (is that really a verb?), why would someone say this?

This seems to happen when management realizes it will take longer to test something than they initially planned and/or some client demands a product sooner. The most recent occurrence came when management didn’t communicate the expected release date and freaked at the estimated test duration. My response was you can have the product whenever you want but let me tell you what won’t be tested. This elicited the response “no we don’t want to not test it, how about we… throw someone else in to help QA it faster.” Clearly someone hasn’t heard of Brook’s law.

Adding manpower to a late software project makes it later.

Fred Brooks

Brook’s Law is a term coined by Fred Brooks in his book The Mythical Man-Month which states “adding manpower to a late software project makes it later”. It also appears the person saying this doesn’t understand what Quality Assurance (or QA) means.

If the role of software testing is to help the team understand vital information about the product and you bring in someone who doesn’t understand how this is accomplished, the value both will be providing is diminished. You slow down the primary tester as they coordinate with the person being brought in as work is divided up based on skill and comfort level. Configurations of different machines, problems local to the users and a whole host of other problems can crop up as well. In other words it takes time for people to become productive on a project.

Anyone who does time-based work (has a deliverable) can tell you it’s helpful to know when you need to be done. Crazy, right? Testing a product is a never ending journey but specific periods of testing can end, for example when the product is needed in the field. There will always be more testing to do but you don’t always have time nor is it always a good use of resources. Dates can help. If this statement comes up often either Management or the Team has problems communicating with each other about when things need to be done. Setting dates isn’t a sure fire method since dates can change but so can the decision on what needs to still be tested and what’s acceptable risk for the field.

While it’s possible to get an additional person to add some incremental value into a project (they might have some unique perspectives that can help, especially if they are subject matter experts) it’s going to take them awhile. Don’t assume “throwing someone else in” will do anything other than make the testing duration longer.

Testing Idol Worship?

With software testing I’ve found it important to identify a few key experts in the field to see what they’re saying, doing, reading, etc. in order to learn and expand my testing thinking. Maybe it’s the size of the industry, the lack of “basic” testing education but there seem to be a number of ways to get trapped in the “hump” of a testing career. Perhaps all industries are like this and this is the first time I’m experiencing it?

Luckily Twitter makes following experts very easy as do blogs. I’m excited so many testers blog! To the untrained eye it might look like I’ve got a case of testing idol worship. A year ago or so ago I was writing a lot about James Whittaker and now all I seem to be doing is writing about James Bach. (I wonder how my Google page rank looks when searching for the name ‘James + software testing’…?)

Don’t worry it’s not testing idol worship, at least I don’t think it is. It’s more an exploratory way to learn about the testing body of knowledge and discuss (with myself ha ha ha…. sad face) the things going on.

Bear with me and enjoy the ride.

The Role of Testing by James Bach

The following is a summary of the essay The Role of Testing by James Bach from the book Amplifying Your Effectiveness: Collected Essays.

The essay goes like this: After not liking his time as a developer James thought being a testing manager would provide more wiggle room, since testing is a little more vague than programming.

James says “I used to think that the role of testing is to find problems.”

Finding problems was easy but not satisfying in the long run (I can attest to that) so after one of the managers at Apple sent around a book called Quality Without Tears he became a bug prevention, zero defects tester.

Then James says “I used to think that the role of testing is to assure quality.”

Yet testers can’t really assure quality because perfect quality is an inherently unreachable goal with many dimensions to it. Testers don’t create quality, most of the time its not in their power to perform. If testers assume the mantra of quality gatekeepers then it’s likely the rest of the team will take less responsibility for quality. QA will be there to assure it and when they can’t it’s their fault. Additionally many QA people naturally exert control by defining processes and auditing compliance which often slips into moralizing about quality until all anyone can agree on is good quality is good and bad quality is bad.

At some point James’ manager let him in on a secret: testing is about the search for risk, not perfection. Finding important problems fast is the game, not covering the entire system in detail. To do so you have to understand which parts of the product really need to be tested which can bring the tester more into alignment with the rest of the project.

Then James says “I used to think that the role of testing is to analyze risk.”

Yet risk is an abstract (although important) concept. To some it sounds like a way to focus on the negative – what’s risky, while the business takes the opposite approach – encouraging risk (entrepreneurial).

So James says “Today, I think the role of testing is to bring vital information to light in support of the grand mission of creating great products and running the business. That includes finding problems, assessing quality, analyzing risk and in any other way helping the team to understand what’s going on.”

This was my favorite article from the entire book because the subject of the “role of testing” doesn’t seem to be widely understood today, 12 years after this book was published. The simplicity of James’ statements show how he embraces each idea and then discards it, showing us how he learned a little and moved on – lessons learned, if you will.

I wonder, does James still think this way? How long did it take before he realized a certain “perceptual role” didn’t work and began looking for another? From the limited experience I’ve had studying his work I can see how this evolving view influenced his career. For example James has written articles on (heuristic) risk based testing. His Rapid Software Testing course uses a combination of these “perceived roles” to focus only on the important parts that matter – increasing the time you can spend on testing – hence the titled ‘Rapid’.

I think in a way James is right, our role as testers is to light the way. Although saying that sounds too abstract, too feel good. Whatever the right phrase, helping the team understand the product seems like a good direction.

The book is called Amplifying Your Effectiveness: Collected Essays edited by Gerald Weinberg, James Bach and Naomi Karten. It’s a real quick read and you can pick up a copy cheaply on Amazon or eBay. At 140 pages or so it was super quick to read. According to the book this essay was adapted from an article called “Rethinking the Role of Testing for the e-Business Era.” Cutter IT Journal, Vol. XIII, No. 4.

Rapid Testing Intensive Confirmed!

(Stolen from the Rapid Testing Intensive site)

It’s official I’m booked for the onsite Rapid Testing Intensive with James and Jon Bach at the end of July on Orcas Island in Washington. According to the website this testing intensive will be based on “… Session-Based Test Management and Rapid Software Testing methodologies” and will “…allow you to see how the modern theory of testing meets practical work.” Sounds like a blast.

There are 10 onsite and 42 online participants as of 4/2/12 and one of those onsite partcipants is Robert Sabourin. I was in his “Using Visual Models for Test Case Design” class last year at StarWest so it will be interesting to work side by site with him as well as a few of the other participants.

As I said in my prior post my goal is for: “Experience and feedback on modern testing methodologies!” Can’t wait.

Are Testing “Schools” a Good Idea?

There has been some controversy with Cem Kaner announcing the Context-Driven School of Testing will no longer be called a school. Cem believes (as I understand it) calling something a “school” is too divisive resulting in an exclusionary system that might possibly ignore people with great ideas who don’t necessarily identify themselves as being in the school.

Division for classification purposes seems to have worked for numerous scientific branches and to me doesn’t seem like something to worry about. I understand how those placed in a school might find it offensive but that doesn’t mean the classification should change. It might change how one talks to other testers (i.e. not making it a divisive issue) but I personally don’t feel like I’m trapped in one school and am blind to other peoples ideas.

I’d prefer to hear a debate between James Bach and Cem Kaner but instead a video has surfaced between James and Doug Hoffman called “Are Testing “Schools” a good idea?” from CAST 2011:

As someone who is constantly exploring the testing “body of testing knowledge” I look first to those who subscribe to Context-Driven schools but even those who subscribe have very different interpretations / views on testing.

Perhaps at some point I’ll understand the greater implications of this but as of right now it seems like a far off issue.

Bach Brother’s Rapid Testing Intensive

When I was at StarWest in October of last year I had the good fortune of running into James Bach at the end of the day. I participated in a Rapid Software Testing class with his partner in crime Michael Bolton the prior day and sneaked into James’ Critical Thinking class earlier this day. He was approachable so I asked what books he recommended for aspiring testers (an easy opening), then when he’d be giving another talk on Rapid Software Testing in the US. I told him I liked the videos of his open lecture’s (I’ve blogged about them here and here) and somewhere during the discussion he mentioned a plan to setup a rapid software-like testing session near his home in Washington.

That testing session has been announced as the Rapid Testing Intensive taking place from July 23rd at 6:30pm through July 28th at noon on Orcas Island, Washington. I’m continuously trying to convince the company I work for the session is well worth the expense to go in person. There are two options, join in person or join online. I’d prefer physically being there instead of virtually for the easy of communicating and the overall experience.

The one day Rapid Software Testing session at StarWest was very inspiring but also brief. As Michael put it… the Rapid Software Testing course is normally a week long that gets crammed into three days and for StarWest is crammed into one day.

I’m not quite sure what to expect of the testing intensive but from what I can tell my goal for attending would be to practice using the rapid software testing methodologies with those who created it. I’m interested to see how they put their “modern theory of testing” into action so I can take it back to my company and apply it.

All of the Rapid Software Testing materials are available free online. It’s one thing to read the materials and practice what you absorb but its quite another to work with those who built the method/theories to put everything into practice. The day I spent at StarWest in the Rapid Software Testing class was great but we were only able to cover a small amount of the material.

I’ll need to do a few things in preparation:

  • Review the Rapid Software Testing material we covered in class
  • Read the material that we didn’t cover (lots)
  • Review Session-Based Test Management
  • Figure out the testing tools I’ll need
  • Maybe start on the BBST coursework?
  • Plan the travel details!
After the class I’ll be a bit more informed on what does and doesn’t work and what assumptions I’m making today when I test. This session will also be a test in critical thinking and afterward I hope to be more empowered to do research into the testing abyss to find my own path.
Perhaps I should say my goal is for: “Experience and feedback on modern testing methodologies!”