Shifting Computer Interfaces

I was listening to the Cheeky Pints podcast with Ben Thompson and Jon Collison, who were discussing agentic commerce and how it might play out. In a nutshell, agentic commerce is about using chatbots to handle your purchases:

Collison: And so it’s like, “Please book me a honeymoon in Japan and all the activities”, it’s like no one actually does things that way.

Whereas actually you should go from the bottom up in some very basic building blocks where step one is just replacing filling out web forms, that’s an activity that sucks, no one likes it. And so imagine you find the winter jacket you like and you copy the URL into ChatGPT and just say, “Please buy this for me”, and that’s a much better experience than going and clicking around a site you’ve never been to before. So there’s just the agents doing the kind of tool you saw on your behalf and everyone can create that. Maybe it clarifies there’s multiple colors, “Which one do you like?”, but it’s just replacing filling out form fields.

Thompson: This is, by the way, one thing that I am very — a lot of people are skeptical of this, but I am very optimistic about — which is, I call it just in time UI.

Transcript from Stratechery.

This conversation struck a nerve because just-in-time user interfaces mean we have an entirely new class of users for our software.

The Interface Has Shifted

Occasionally I’ll take a picture of a product and ask Gemini to identify it. If Gemini offers product links, I’ll click them and even buy stuff. I did this a few days ago because I needed a specific wrench for tightening spark plugs and I didn’t have the right tool. Much like how Googling came to dominate how we discovered products, chatbots are replacing that. Only they can go much further.

At work I use Jira more now than I ever have. I just don’t use the Jira interface as much. Instead, Claude Desktop is my interface and Jira works as the backend storing the data that I need to retrieve. I ask questions directly about the underlying Jira data, generate reports, pull tickets, and navigate directly to Jira’s interface as the last step if I must.

I’ve already shifted from being the operator of certain apps to using them indirectly through a chat interface.

Can an agent operate my website?

As I said above, I’ll use product links in a chat to make purchases. However, no agents are helping me fill in forms or complete the purchases.

This got me thinking: if the first useful building block of agentic commerce is filling in forms, could an agent complete the membership signup for this website? More importantly, could it do it without detailed instructions?

To do this, I needed to give my harness (Codex) two tools:

1. A web browser
2. An email inbox

Since Codex already has a built-in browser, it was just a matter of installing the Mailinator CLI for email inbox access:

npm install mailinator-cli

Then I gave my agent a request and hit enter:

Sign up for a membership on my website Kenst.com and sign in. 

In less than four minutes, it came back and said Done. When I asked Codex for a play-by-play, this is what it said:

1. Opened the membership flow on Kenst.com.
2. Selected the free membership option.
3. Registered with a unique public Mailinator address: kenst-vibium-0722@mailinator.com
4. Checked that inbox through the Mailinator CLI and found the Ghost confirmation email.
5. Retrieved the message and extracted its “Confirm signup” link.
6. Opened the confirmation link in the browser, which completed the membership signup and automatically signed the “new member” in.
7. Verified success by confirming that Kenst.com displayed the member-only welcome page and replaced “Sign in” with “Account”:

To doubly confirm this wasn’t just smoke and mirrors, I logged into my Ghost dashboard and could see the new user sign up:

What the Agent Figured Out

If we think about this experiment in terms of what we gave the agent and what it was able to figure out without detailed instructions:

The request was one sentence, but completing the goal required several decisions. The agent had to infer that “Subscribe” was the route to membership, identify the membership options, and choose the free one. (Given the recent “creative choices” advanced models have made, I’m glad it didn’t select the paid membership and then try to steal money.)

It also had to recognize that submitting the form wasn’t the final outcome. That there would be an email component and a confirmation workflow that would require the use of the Mailinator CLI to continue toward the goal.

A New Class of Users

Going back to the podcast, Collison’s point was that agents should start from the bottom up by automating boring things like filling out web forms. This experiment showed that an agent (with the right tools) can operate a Ghost blog membership form. It’s very plausible the next time I need to find a wrench Gemini or another chatbot could take me further by logging me in and adding the product to my cart.

Website design has long been optimized for people. We’ve made great strides as an industry to make websites more accessible for people. Now we’ve got another class of users to think about: agents.

The Interesting Part

The interesting part of this experiment wasn’t that Codex could fill out a form. Browser automation has done this for decades. The interesting part was that I simply described an outcome, supplied access to an inbox, and let the agent determine the steps to get there.

Similar to my experience with Jira, Kenst.com’s interface didn’t disappear, but I stopped being its direct operator. If that pattern becomes more common and our software is increasingly driven through an intermediary like a chatbot or agent, there are challenges ahead. We will still need to design products for people while accounting for the agents acting on their behalf. That shift will have consequences for usability, accessibility, identity, testing, and what it means for an interface to work.