I’m Speaking at TestFlix

It’s true, I’m speaking at TestFlix on November 28th, 2020. You should sign up to join; it’s free to register!

I recorded and submitted my 7 minute talk on “Using Test Idea Catalogs for Better Testing”. The premise is:

Testers can develop a set of tests or test concepts for a specific object or risk and re-use them in similar projects or products. Catalogs come in many shapes and forms, they can be lists or more detailed. They can be public or private. They can be developed by individual testers or as teams within companies. But they all help you test better!



I plan to write more on the topic of Test Idea Catalogs in the near future but I mentioned a few in the presentation that I’d like to call out here:

The Woolsey Fire came too close

Today, when you approach Calabasas from either direction on HWY 101 the first thing you notice are the black rolling hills. As far as the eye can see there are visible signs of the struggle that came with the Woolsey fire. Lucky for us changing winds and a strong first responder presence stopped the fires from getting to our place.

We woke up Friday, November 9th to a stream of text messages starting at 1am and carrying to 6am from friends asking if we were alright, had we been evacuated and did we need a place to stay? As we slept in our hotel room in San Francisco it went unknown to us there was a fire sprawling around the coast of Southern California. Mandatory evacuations had come in the night and the towns next to us were evacuated: Thousand Oaks, Oak Park, Agoura Hills and now Calabasas. We promptly responded.

Hundreds of miles away and previously occupied with San Francisco’s poor air quality due to 2018 Campre Fire that burned the city of Paradise, CA (to date CA largest fire ever), we started searching for information and watching the news. The fires weren’t as bad as the ones raging in Northern California but they were closer to home. Extremely close to home. Now it was a waiting and watching game. What could we find out? How close were we to loosing our home and everything in it? What would the Woolsey Fire do? Which direction would the Santa Anna push the fires next? We left SF and travelled towards Stockton but were unable to get away from the concern for our own place and the poor air quality of the Camp Fire.

Fast-Spreading Hill Fire Forces Evacuations In California's Ventura County

Despite the TV news coverage both Twitter and Instagram were our best sources of hyper local news with hashtags for our particular area being the most helpful. Go figure. Between local news reporters tweeting out details as they drove the streets and connections to random neighbors or friends of neighbors we got sporadic word of mouth updates throughout the time. Stories of nearby buildings on fire fed rumors that turned out to not be true (luckily).

A day later the winds changed direction and the fire turned away from Calabasas and headed south to the coast and towards Malibu. (I later heard reports of Malibu evacuations taking people 4 hours to travel to less than 20 miles to Santa Monica). A sigh of relief came, if only because it meant a chance to contain the fires near us. Unable to return home we stayed with family in Santa Clarita. We stayed away from home for nearly a week due to road closures, poor air quality, a lack of power and lack of internet but we were safe. In the end so was our place.

Two weeks later and things have mostly returned to normal. Our home just barely survived, with parts of the creek near us burned and a hotel two blocks away closed due to fire damage. There’s still some lingering smell of smoke but inside our home, our new air purifiers have mostly removed the reminder. Others weren’t as lucky and we almost weren’t either. It’s good when luck is on your side. Either way this fire came too close for comfort.

(In hindsight, writing about these events as it was happening and after helped me cope with some of the stress.)

Rebooting the AST’s Webinar Series!

Last year, after failing in my bid to become a Board Member, I agreed to run the AST’s webinar program. Funny thing was I already had a small list of people and topics I wanted to learn more from / about based on conference and podcast talks. (When something intrigues me I take notes to research later.) Now I have an opportunity to track down those people and ask them to (generously) share their time with the wider world of testing!

Since the AST is a non-profit with a goal of building and developing a community of skilled testing craftspeople, anything that falls into the large arena of software testing could become a topic for discussion. It doesn’t always have to apply to an aspect of test automation which is good and should give us a larger pool to draw from. This is going to be a challenge for me. Logistically challenging but also about being inclusive of interesting topics and presenters that I might not be aware of.

For 2018 the goal is one webinar per month. I have no idea if this will be a sustainable pace or if all of my presenters will be able to deliver but that’s part of the challenge.

Anna Royzman is our presenter for January so be sure to sign up. Alan Page will present in February and after that? Who knows!

Having said this, I’m always looking for presenters and topics. Got an idea? Contact me!

A few of the past webinars the AST has hosted are available online:

Introducing TestingConferences.org

For the past few years one of my professional goals has been to attend (at least) one testing conference or workshop per year, mostly because it’s such a great way to recharge and learn what other practitioners are doing. Unfortunately, I couldn’t find a good source of events aside from talking with others, so I pulled together a list on my own. The value of any events list is only as great as the number and quality of events listed so I’ve open sourced the list and posted it online with the hope that it becomes driven-by and representative of the community as a whole (as opposed to what I might like or prefer).

Introducing TestingConferences.org – a simple list of software testing conferences and workshops published collaboratively with the testing community.

I hope this site is or becomes a useful resource for the software community as a whole. I also hope others will help by contributing because that’s the only way it will become better or at least maintain its usefulness. It doesn’t matter if you are a vendor or just a fan of workshops and conferences, please add to it!

A little more detail

I’m not sure how I found my first conference, StarWest, back in 2011 but I did. Every other conference or workshop I’ve been to since has been a referral or recommendation by others, including:

  • The Rapid Testing Intensive workshop (RTI)
  • The Workshop on Teaching Software Testing (WTST 13)
  • CAST
  • STPcon
  • STARWest (I’ve been to this twice I think)

I say referral / recommendation because the primary way this stuff was and is communicated are through the people who’ve attended. Sure you can Google “software testing conferences” or come across an advertisement in a testing publication but those are only useful if you have an idea of what you are looking for. Even if you do those things you might come across the most advertised, most popular but not necessarily anything near you (especially if you are outside the US) or relevant to your particular tastes. To me, that’s sad.

Conferences and workshops are great tools for conferring, collaborating and learning. At least part of your decision to attend a conference or workshop is determined by knowing what conferences are available and where they are located. That’s where this list comes in. There was never any single source of active conferences and workshops; especially with workshops you had to be in-the-know, else just stumble across one that was occurring.

Now I’m asking for your help so we can publish this list collaboratively within the testing community.


Truth is I’ve been developing this site for a while and promoting on Twitter as I added events. I suppose you could say this site has been in stealth mode and this is the official reveal.

See you at CAST 2015

Thanks to my awesome company’s sponsorship and my wife’s love of travel I will be at CAST this year and attending the tutorial “Delivering Difficult Messages” by Fiona Charles. This is both my first time attending CAST and traveling to Grand Rapids, MI so I’m expecting to have a little fun and to learn a lot. As for the conference experience, I’m not sure what to expect.

Over the last four or five years I’ve been to a few different conferences and training events:

WTST 13 was my first-ever peer workshop and the last trip I made. In contrast to large conferences like STAR and STPCon where several hundred or even a thousand people crisscross each other as they go to any number of tutorials or talks, the peer workshop format was limited to 20-25 participants in a single room. There were a few presenters who spoke along a similar theme but it was the participants who drove the discussions until everyone was satisfied. It was certainly a unique experience.

Comparing workshops to conferences may not be fair due to their different approaches but my goal for attending them is the same: to learn something new and useful (apply to my job or company) and interact with my peers.

My understanding of CAST’s format, even before going, is it’s a small-ish conference attended by few hundred conference-goers, most of whom are AST members. Although a few hundred participants is still a good-sized event, I’m hoping to be able to interact with a few (or a lot) people. Of the few conferences I’ve attended the biggest impact has always been the interactions: meeting others, sharing problems, being challenged in my thinking, trying to explain something, reference recommendations, etc.

Aside from the one tutorial I’m signed up for and a welcome get-together in the morning each day I don’t know what else there is planned. There’s no schedule available, yet and until there is I won’t be sure what to expect.

Enough about my expectations; who else is attending and what are you looking forward to?

Winning the #KaepernickingWorldwide Contest

In July of last year my then-girlfriend and I spent three weeks traveling mostly around the Philippines with a few days in Hong Kong and Singapore. We made the most of our trip to Singapore when we randomly stayed one night at the Marina Bay Sands hotel. (When I say randomly I mean we booked the hotel and flight the day before we left!) We chose the Marina Bay Sands thanks to my future brother-in-law’s recommendation of it looking like a spaceship (and who doesn’t love spaceships?).

The hotel boasts the world’s largest infinity pool, essentially stretching from one end of the roof to the other. In person it’s such an amazing experience to be sitting in the pool at night (or day) looking out onto the Singaporean skyline and beyond. The flip side view is equally amazing as you look out into the very, very busy harbor and the gardens below. As “tourists” we probably took more photos from the Marina Bay Sands than from the rest of the Singapore trip combined (even counting our trip to the massive aquarium).

Both my wife and I are HUGE San Francisco 49er fans. She, more than I, enjoys posting and following people on Instagram but we both follow the San Francisco 49ers quarterback Colin Kaepernick’s account. I think it was August of this year, when we noticed he was holding a photo contest called #KaepernickingWorldwide. Participants would take a photo of themselves or someone they know “Kaepernicking” (see the featured photo), post it on Instagram, tag Colin Kaepernick, #kaepernickingworldwide and their location. We were excited because we had a photo from our stay at the Marina Bay Sands where I was Kaepernicking in the pool. Neither of us figured we had a chance to win.

A few weeks after the post went live we got a direct message from Kaepernick (Kap) saying we had won! I think we jumped for joy for about 10 minutes before we responded saying thank you and providing our shipping address. A follow up search on the hashtag revealed about a half dozen or so winners – so we joined in on the fun. At the time it wasn’t entirely clear what we had won (it looked like a bobble head, some t-shirts, socks, and wristbands), we were just happy to be part of the contest!

Roughly 9 weeks later, we arrived home from our wedding in Napa to find a package sitting at our doorstep from Kap. What a great wedding gift! We were amazed when we opened it:

Among the great items Kap sent us was one of his Jerseys signed along with the tag #kaepernickingworldwide. Wow! This isn’t the kind of Jersey you can buy online (we have a few of those), this is the heavy duty version with the short sleeves he wears on Game Day. Again, wow! Also included were a signed copy of GQ Magazine, signed football card, a signed action figure from his Nevada days, some clothing and a few wrist bands. We put some of the items together into a case and hung it on the wall.

As 49ers fans we’d like to formally say thank you to Colin Kaepernick. Often being a a fan of a team or a person (sports, business, tv, whatever) is a one-sided affair so thank you for briefly changing the dynamic. We are even bigger supporters than before. Go 49ers!

Cross published on Medium.

See you at WTST 2014

The Workshop on Teaching Software Testing, abbreviated as WTST and I think pronounced “what’s it”, is coming up at the end of January 2014 in Melbourne, Florida near the Florida Institute of Technology campus. (You maybe have seen a few of Cem Kaner’s posts.) Just recently my application was approved and I’m excited to be attending as a non-presenting participant.WTST is a LAWST-style workshop which means unlike regular conferences it’s limited to small group of active participants, some who present and everyone is expected to engage in discussions by asking question, providing their experiences and even thoughtfully arguing with the presenter. Discussion is the entire point of the workshop.

This year WTST is focused on designing and teaching advanced courses in software testing, something I’ve had an interest in for a few years now. Teaching software testing was the driving force to me becoming an Instructor for the Association for Software Testing’s BBST classes (which I completed just recently).

This year the workshop will be broken into two sections: The first is the typical weekend-long WTST workshop and the second is the Domain Testing Workshop held for five days immediately after the traditional WTST. I’m excited to be part of a pilot course built around The Domain Testing Workbook (see my previous post for more info) and I imagine it will be a fun exercise in understanding domain testing a little better.

Who else is going? Hopefully I’ll see you there. =)

For more information see http://wtst.org.

The Gang and Us

Last night on It’s Always Sunny in Philadelphia the episode we were on set to make aired called The Gang Tries Desperately to Win an Award. It’s on Amazon, iTunes and possibly other places but I bought it and cut out the little section we are in.You’ll find us in the background at roughly 0:04 giving a high-five to the bartender and at 1:11 and 1:35 for our close ups. Ah our 4 seconds of fame; well worth the 10 hours on set. lol.

It’s Always Sunny in Philadelphia Season 12, Episode 3. Enjoy:


On Set with It’s Always Sunny in Philadelphia

To watch our part in the aired Episode go here.

A month ago I was at my buddy Joe’s annual Multiple Sclerosis fundraiser when I saw a silent auction for It’s Always Sunny in Philadelphia (my favorite show).  After some intense bidding to the very end of the auction I was victorious! As you can see below the package came with an autographed poster, autographed 2013 calendar, DVDs, autographed Sweet Dee bobble-head, kitten mittens T-Shirt, autographed group photo and, more importantly 2 walk-on roles.

It's Always Sunny package
The package of It’s Always Sunny goodies!

A few emails of correspondence with the It’s Always Sunny team and a few weeks later we had our shooting date. Call time was 7am on location in North Hollywood at The Federal Bar. At 7am the bar was only a 20 minute hop from the girls place (practically next door when you’re in Los Angeles) so we grabbed some food on the way just to be safe. I’d heard stories from friends about shows running out of or having crappy food for the extras and we didn’t want to take the chance. We arrived on set, checked in with the rest of the extras only we weren’t on the list as extras and then were separated as they called us contest winners. (On occasion they’ll have radio show contests and people will come from all around to visit the show. You are treated differently because you aren’t being paid.)

Check-in sent us to wardrobe where I was given a darker collard shirt and the girl was given a jean jacket to wear with her dress. Naturally she refused to wear it because, well, who wears jean jackets these days? We headed back over to get some food from craft services and waited until we were told what next to do. Ten minutes later one of the crew members came over, gave us some details on the scene and why we were at a bar and told the extras to head to the holding area. As they headed towards the holding area the girl and I were lead onto the set to watch the action.

The crew is setting up the bar lighting. Originally we sat where the ladder stands.

It took some time for the crew to set things up, something about adjustments or last minute changes. From our point of view it seemed like a lot of variables to deal with for maybe 5 minutes or less of total air time. We sat at one of the tables while the crew set up lights, the director ran around looking for shots and many others things unbeknownst to us occurred.

Now either the crew was used to having visitors on the set and/or they were just very nice because everyone kept coming up to us introducing themselves and asking where we were from. (Apparently some contest winners came from more than 20 minutes away!) I wish I could remember their names to give them proper thanks. I do remember we met some very nice lighting technicians, the associate director to the producers (I think that’s his title), the director, a really nice cameraman and others who, like I said, I wish I could remember their names.

At some point after sitting down, during the random meet and greets, we were given some background on the scenes being shot (2 scenes) and given a copy of the script. Naturally I read through it to see what was going on. One of the crew members walked us through the other parts of script including the actors who’d be on set, what scenes were being shot, etc. Turns out for this scene it would be The Gang minus Charlie but including Frank. At some point a few people came out to stand in place of the actors for the lighting adjustments. Then a while later, maybe 30 minutes, The Gang came to the set and did their line walk thorough. Charlie wasn’t in the episode but he was still there watching them as they went through their lines. After they were done with the walk through The Gang came over to our table and introduced themselves. Very cool!

Our drinks from Sudz bar. Sorry the photo is a little blurry.

I wasn’t paying close attention to the time but my guess is it wasn’t until around 9:30 or so until the girl and I got placement at the bar for a scene (mind you we had been on set since 8am or so). One of the benefits of the walk on role was “prominent” placement on camera (in terms of extras), so the girl and I ended up at the end of the bar next to the beer tap and kiddy corner to where The Gang was sitting. How prominent will we be when the show airs? Who knows, hopefully it will look like we are bar patrons and neither of us is just staring at Mac. The entire time I was drinking quality no-name beer and the girl had some fruity drink.

Around 10:30 The Gang came back to set and we started filming the first scene of the day: The Gang has arrived at the bar and are talking about what they see. For some reason Frank has brought a goodie bag with him. The next several takes / scene involves The Gang sitting down at the bar. Done; time for a break. It took that long and we’ve barely shot anything. The main actors take off and the girl and I wonder over to get some food. It’s interesting because there’s so much activity taking place on the set and so little of it requires the main actors.

Just past the bar where the scene is taking place was another bar where the crew had set up craft services tables, chairs for people to sit in, and a video feed for the directors and producers to see the shots they’d filmed. During one of our breaks one of the crew suggested we get this shot:

The Gang also contribute to the scenes through writing and direction

At break time there was food and a chance to sit back and relax and do whatever until the next shot(s) were ready. During one of the breaks we walked outside and joked a bit with the other extras about how much smiling we had to do (as soon as they start filming we have to talk to each other without speaking and always smile). It was like being at the dentist for a long time, eventually your face starts to hurt. A little chit chat to break up the monotony and then back for the next shots.

When it came time for the main shots the girl and I pretended to be talking with another bar patron while The Gang sat across from us, delivering their lines. I didn’t pay a lot of attention to what they were saying because I was trying to “act” like I was having a good conversation which is hard when you are mouthing words but not saying anything. The show shoots 2-3 different angles with 3 different cameras which means repeating the same scene over and over again. During one of those angle changes the girl and I had to move.

We were sitting right where the camera guy is now standing. To the right is the feed for the 3 cameras.

Since I’d read the script I knew what The Gang was saying but it wasn’t until we sat next to the director that we were really able to see the scene come together. After the move we ended up sitting behind the director and Charlie Day, who wasn’t filming but was occupied with making the scene as funny as possible. On occasion he’d stop the actors and give them funny lines or tell them to keep going with a particular line or word they had. Day is a writer and producer of the show and he didn’t seem satisfied unless he laughed at the lines himself. Although the episode is scripted the actors do a fair amount of ad lib; in fact the funniest stuff is off the top if their head, building off each other’s conversations and building off of the script. They do so many takes of the same scene its during those later ones that things start to get really funny / over the top.

Lunch came around 12:30 and we all headed out of the bar towards craft services where we sat in the morning. As we ate the yummy food, we sat and talked with a few other extras about the jobs they’ve had (one girl was on set for 2 days in Miami for the filming of Iron Man 3). Then we described how we got on set – fundraiser winner! We returned to set after lunch at 1:30 but the crew was still setting up so we were escorted back out for photos with the cast. Apparently Charlie had left the set but we were psyched to get photos with the rest of The Gang. As we were walking back towards the cast’s trailers I noticed a Tesla Roadster. As I was drooling over the Tesla Roadster Rob and Kaitlin (Mac and Dee) walked up and jokingly said don’t scratch the car or the owner will get pissed (he was the owner). Rob and I started talking about his Tesla (I love ’em), how sad it was they stopped making the Roadster, if he was going to get a Model S which is when he pointed to Glenn’s (Dennis) Model S right behind it. Sweet.

Then we posed for pictures. Danny DeVito had come wondering up as we were talking with Kaitlin and Rob so we got a photo with them:

Danny DeVito, Mai, Rob, Chris and Kaitlin

After the first photo the girl and I joked to Rob about how she always does the stereotypical “Asian peace sign” pose. He laughed and mentioned a time some asian fans came up to him and wanted to take a picture in his car. They did the exact same pose. Danny took off and Glenn came up so we got another photo. This time everyone did the “Asian peace sign” pose:

Over featured image with Glenn, Mai, Chris, Kaitlin and Rob doing the “Asian peace sign”

After the photos we headed back into the bar for some final shoots with the extras. By 2:30pm we were tired and said goodbye. It was a great day on the set of It’s Always Sunny in Philadelphia.

Taking the RTI Online

In February I attended an online training course where participants test a software product using the Rapid Testing methodology called Rapid Testing Intensive (RTI) Online taught by James Bach. I found it to be a great way to test a product, get feedback on your work, build a software testing portfolio and learn about the Rapid Testing methodology.

Last July I took a similar in person training course appropriately called Rapid Testing Intensive Onsite. I meant to write about my experience but never did so allow me to describe it now:
The onsite version was an intense four and a half days of lecture, learning, testing and other team activities. From survey testing, to group stand up presentations, to the occasional after hours (with beer in hand) dice game it was a week of mental challenges with quite a bit of fun mixed in. After some encouragement from a few twitterers I shared my notes in the form of a live blog: (day 1, day 2, day 3, day 4, day 5).
During that time I was the “lone tester” in my company and taking a week to work on a team with other testers from around the world was a welcome change and an enjoyable experience. When a team member found something interesting or became confused the rest of the team became involved in the discussions which lead to new ideas about where to test. If someone didn’t clearly understand something someone else in the group could help. All this team work lead to some exciting discoveries.
Coming back to my original story I can do a little bit of comparison:

The online version was much more concentrated than its onsite counterpart but maintained all of the important aspects with an opening lecture, followed by a 90 minute testing assignment and then a break. During the break James would do the assignment himself and when the participants were back he’d show what he did, explain his approach and then review the work of those who were brave enough to ask for it.

Asking for your work to be reviewed live (webcast) can be a little intimidating. I remember doing the stand up presentation for my group onsite and I made a lot of mistakes in front of others but the result was I learned about safety language. This time around I knew mistakes would mean I’d learn more so I didn’t hesitate to ask. After the feedback session there was a little more lecture and then the day is done. This was the schedule for the rest of the course except with each day the assignments build off of one another and get deeper into the product.
Having essentially taking this course twice (first onsite, second online) my approach was to focus on understanding the assignments while also building examples of my work that I would be proud to share afterwards.  I used James Bach’s Heuristic Test Strategy Model to generate ideas for covering the product we were testing that would go into my Product Coverage Outline. Through both experiences I got new inspiration / ideas on how to organize and document my deep / combination testing. I played around with the way I take notes and continued to experiment with a method James calls concise documentation (however I prefer the catchier term “lean documentation”) which means there should be no fluff in your documentation, just the important parts.
Perhaps the most visible output from the RTI online are the work examples you build while documenting your testing – the Product Coverage Outline, deep testing matrices, testing notes, etc. These documents, after some cleaning and framing, will become the basis for my software testing portfolio – a public example of the work I’ve done and am capable of doing.
Not so visible are the truly important parts from the RTI online: learning about testing and deliberate practice. After all you can only get better at something if you practice it. Rapid Testing teaches us how productive and exciting testing can be by focusing on personal skill and the thinking part of testing. In my experience testing has always been about documentation (think test cases) and not about thinking / questioning what needs to be done. Every time I’ve taken a course in modern software testing practices (like RST) I feel like I start to understand more of the things that effect what I do and I start to question the assumptions involve. With the RTI I get to learn and practice which makes me feel like I’m growing and getting better.
I’ve found the product of those learning experiences, of the direct feedback to my work has improved how I test at work and how I view my product and the value of my labor. I’m not satisfied with where I’m at now, I might never be, but I do understand how far I’ve come and how far I have to go. See you at the next RTI.