AI and the Declining Cost of Trying

I'm experimenting with using Claude CLI to help me tackle work that I might not normally do. (Substitute Claude CLI for your favorite agent harness).

We have a huge backlog of tickets to automate at work. Once a week, I review these tickets to understand the scope of work being requested and to find opportunities to reduce the size. That’s when I noticed a good amount of tickets in the backlog were component tests.

Component tests aren’t something SDET teams usually tackle. Along with unit tests, they are typically written by developers. A common pattern I’ve seen is that developers build developer-facing tests like unit, component and even some integration tests while SDET teams build customer-facing tests (integration and UI).

As a manager, I'm of the mind that SDETs should be capable of adding tests wherever they are needed. However, from a more practical standpoint I know that writing tests closer to the code requires different skills. (My SDETs have also told me repeatedly they expect developers to do that work.)

This seemed like a perfect opportunity to experiment with using Claude to write tests. One day I gave Claude several of these Jira tickets and asked it to write the component tests. (The Jira tickets I used spelled out in a scenario how to use functionality, so that helps.)

I spent maybe 30 minutes in total on this work. I supplied Claude with two Jira tickets, gave some additional context about where the code lived and scoped the work to produce component tests. Then I tested them from the command line, flipped their assertions to make sure they failed and then put up some pull requests. Claude also identified some existing component tests that were bad. One component test had the assertion:

// This code will never find a problem

expect(true).toBe(true);

Guess what? Claude did a pretty good job.

Now this isn’t a post about how great AI is at writing test code or how Claude is the best model. I’ve seen AI produce some terrible test code. Yet it was interesting for a few reasons:

1. This is work that both my SDETs and I would have avoided either due to a lack of skills or time to get it working.
2. This was a fun use of my time. Even if it didn’t work out and I wasted 30 minutes to an hour, that would have been ok.
3. This shows the sort of the leverage that can be had and the new work that can be done. As a manager, I like to be able to push my team to do things just outside their comfort zone. This is one of those things I can push on because I know it’s possible.

It’s this last point that I keep coming back to. This experiment did not convince me that component tests should suddenly become a focus for my team. It did convince me that some work we avoid is now more approachable than ever.

Those lessons are different.

AI agents lower the cost of trying something. They can help you cross a skill gap, explore an unfamiliar code base or validate whether a type of work is harder than it looks. That doesn’t automatically make the work worth doing.