Nobody Decides to Build a Replacement. They Default Into One.

Replacement is rarely a decision. It's what you ship when you build for a job you've never done.

I've written before about why replacing people with AI is a bad idea — morally, operationally, and for the culture you're left holding afterward. I still believe all of it. But the moral case, true as it is, lets builders off too easy. It treats replacement as a choice someone sat down and made. Most of the time, it isn't. Replacement is what you ship by default when you build for a job you've never done and a person you never talked to. That's not an ethics problem. It's a product problem.

Watch how most AI agents get scoped. Someone points at a role — support, ops, sales development — decides it looks repetitive from the outside, and builds a tool to do "the job." Notice what never happened: nobody sat with a person who actually does that job and watched what the work really is. So the product gets built around the visible, legible parts of the role — the parts you can see from a conference room. The visible parts are exactly the parts that look replaceable. The judgment, the context, the thousand tiny decisions that make the role actually work are invisible from the outside, so they never make it into the spec. You end up with a tool aimed at the shell of a job, marketed as a replacement for the whole thing. Nobody sat in a meeting and decided to be reckless with someone's livelihood. They just built for a caricature of the work instead of the work itself. Replacement wasn't the intention. It was the default.

Here's what makes this specific to AI — and why I think a lot of builders are about to learn it the expensive way. Most software automates a shared process. Payroll runs the same way for everyone. A CRM logs a deal the same way across a thousand companies. The process is legible and roughly universal, so you can build it once, from the outside, and it works. An AI agent is not like that. How a person uses AI is deeply, stubbornly personal. It slots into the specific way you think, the order you do things, the judgment calls only you make. The whole world is quietly working this out right now, everyone's relationship with these tools is a little different, because the value shows up in the seams of how each person already works.

I learned this on myself. When I started building my own AI workflows — and later a Socratic agent of my own — the useful part was never some generic "operations" function. It was buried in my processes. The exact way I triaged a task. The specific thing I did over and over that I could finally hand off. The streamlining only made sense because I knew, from the inside, what the work actually was. Nobody could have built that for me from a requirements doc. They'd have had to watch me work or be me. That's the tell. If the value of a good AI agent lives in someone's personal, invisible process, you cannot build a good one without getting inside that process. And you cannot get inside it from the outside.

So here's the question I'd put to anyone building an agent for a role they don't personally hold: are you building with the operators, or building your idea of them? If you're building an AI agent to support operations and you're not talking to operations leaders, not surveying them from a distance, actually working beside them, you are not building support. You're building your assumption of what support looks like, and your assumption is made almost entirely of the visible, replaceable-looking parts. You'll ship a replacement and call it an enhancement, and you won't even have meant to.

The builders who get this right aren't more ethical than the ones who don't. They're just doing the actual product work: talking to the person, watching the real job, and finding the specific, personal places where a tool makes that person better instead of unnecessary. Enhancement isn't a nicer intention than replacement or fixing a problem that does not exist. It's what you get when you did the homework. Replacement is a default, the shape a product takes when nobody in the room has done the work or talked to someone who has. AI agents raise the stakes, because the value of a good one is personal, and personal things can't be specced from a distance. If you want to build something that supports people instead of erasing them, the move is almost boringly simple: go talk to the people. Watch the real work. Build for the parts you can only see from the inside. That's not a moral position. It's just how you build a product that's actually good.