Not everything should cost a token: the case for deterministic AI

A team building a simple morning job—pull a metrics API, reshape some JSON, drop it in a table—decided to let an agent handle the whole thing. It worked fine on day one. Then the bill showed up. Every single morning the setup fed a few thousand tokens of raw JSON into a language model just to reformat it, work a five-line script would do identically every time. Worse, the model sometimes botched the reformatting, because it's a probabilistic system doing a job that has exactly one right answer. And all that raw data sitting in the context window crowded out whatever actual reasoning the agent was supposed to be doing.

This isn't a one-off screwup. Ivan Burazin, founder of Daytona, described a colleague uploading a screenshot to Claude and asking it to retype the text on it, burning tokens on something Preview does for free on a Mac in two seconds. Small and silly on its own, sure. But multiply that habit across thousands of daily agent calls and it becomes a real number on an invoice. The underlying mistake is treating a language model like a universal runtime: if you can phrase a task as a sentence, you prompt it, even when the task is a cron job in disguise.

The fix, as laid out in the piece, is to sort every task into one of two buckets before deciding where it runs. Anything that needs judgment—summarizing, classifying, weighing ambiguous options—belongs with the agent. Anything that has to run the same way every time—scheduled pulls, API calls, storing and querying data—belongs in a deterministic app. Route the wrong kind of task through a model and it picks up three bad habits it never should have had: it gets non-deterministic, it gets slower, and it starts costing money per run instead of running for free.

One mistake gets singled out as especially common: using an agent's memory notes as if they were a database. It feels convenient, since the note's already sitting there for the agent to read, but notes have no schema and no query layer, so pulling one field means dragging the entire note into context. Structured, high-volume data needs an actual database; notes should stay for genuinely unstructured stuff like preferences or the reasoning behind a past decision.

The practical example given involves a churn-analysis workflow: build the app and database once, automate the routine status updates with a zero-cost cron, and save the model's attention for the one weekly question that actually needs judgment—why are these specific customers at risk. Everything else runs as ordinary code. Build the same workflow entirely inside a model instead, and you'd be paying tokens on every row, every single run, forever.