Between us and the machine

Every piece of software I've worked on since 2012 has had the same basic contract: it waits. You open an application, give it input, look at the result, decide what to do next. The system underneath can be enormously complex, but the interaction pretty much follows this principle all the time. You act, the computer responds.

Billions of tokens in AI usage later I came to realize that agents are breaking this very fundamental contract. These days I can hand off a piece of work, close the laptop, and come back to something that didn't exist when I left: a problem investigated, a codebase modified, tools operated, decisions made, incidents solved. People describe this as software getting smarter, and it is, but that's the boring part of the story. The shape of software itself is changing in very interesting ways. We've spent decades designing around moments of interaction. Now agents force us to design around continuity.

The context, history and identity of a long-running piece of work should survive the agent that worked on it. My team and I should be able to inspect it, redirect it, hand it to another agent, pick it up later with a full almost-deterministic understanding of the intent, the decisions already made and the relationships between each piece inside of it. You could argue this is just documentation. In a way it is, the same way git could be seen as just a better changelog (of course it’s way more than that). But you get the point: the record is stopped from drifting the day it becomes engrained as a core mechanism. This needs to be the same kind of object: not a description of the work that someone keeps honest manually, but the thing the work runs on, where the agent resumes from the record and decisions are enforced instead of described. That structure matters more than any individual model could.

There's this depressing public fantasy of where this goes in which the human eventually vanishes completely from the picture. The machine understands the objective, does the work, verifies its own results, and keeps going indefinitely. Every additional human interaction is friction to be optimized away. Eventually it's just machines talking to machines. And maybe some work will run like that. But I believe most work that matters shouldn't.

The point was never to remove people from the system. It's to move human attention to the level where it's worth the most. This is often repeated as a cliche, but I think it does hold some ground. Good automation should be able to act on its own while staying understandable to the person who delegated. That takes more than an approval button or nice code diffs. I want to see what the system thinks it's doing, what information it used, what it changed, what it's unsure about, and where its authority ends. Control also doesn't mean watching every action in real time, because that isn't delegation at all. It means setting boundaries up front, getting my attention at the right moments, and being able to reconstruct what happened after the fact. This might be dangerous and frowned upon, but I believe the systems we'll end up trusting won't just ask for permission. They'll earn autonomy the way people do: by being competent for long enough.

Capable models have made everyone obsessed with raw capability. Where does it place in benchmarks? Can it write the code? Can it use a browser? Can it work for a week unattended? But nobody uses a model. We use harnesses built around models, and the same model can be perfect in one harness while failing miserably in another. A great model doesn't save a bad harness. The resulting tool can be powerful and still be unpleasant. It can be autonomous and still be confusing. It can finish the task and leave you less certain than before.

The interface between people and intelligent software deserves the same care we once gave the best desktop applications and physical tools. Latency matters. Typography matters. Failure modes matter. Being able to read the state of the system at a glance matters. Good infrastructure tends to disappear beneath a simple interaction, but simplicity is not the absence of machinery. It's machinery arranged with care. As agents get more capable, design gets more important for a very simple reason: we need design to make increasingly complex systems feel calm, legible, and dependable. And it goes way beyond just good harnesses.

This is the intersection I keep getting pulled toward: product design, agents, automation, and systems infrastructure. Part of it is plain practical work, connecting tools, automating tedious operations, keeping a company's context from evaporating. Underneath that sit harder questions. What's the right boundary around a piece of work? What should persist when a process, a machine, or a model disappears? How does software act for us without quietly starting to act instead of us? How do we build trust that compounds over years instead of spending it on short-term convenience?

There won't be one answer, and there won't be one universal interface. Many of the important primitives will look too small at first. A terminal multiplexer is ostensibly about keeping a few shells alive. Version control is ostensibly about storing changes to files. A browser is ostensibly about displaying documents. The systems that endure are rarely confined to their first description. They give work a new place to live.

We're at the start of another one of those transitions. Software is moving from something we operate moment by moment to something we keep a working relationship with. Making it capable is the easy part. The hard part is making the relationship durable, understandable, and ours.