How I Cut an AI Agent's Token Use by 94%

Here's a workflow most people building with AI agents will recognize: you write a natural-language "skill" that tells your agent to check some sources, filter out recent stuff, and draft something new — in this case, a LinkedIn post pulled from an old blog entry. It works. But every single time it runs, the agent has to re-derive the whole plan from scratch, burning tokens on steps that never actually change.

That's exactly what one developer noticed after running such a skill daily inside Codex for months. The skill always searched the same sources, built the same inventory, applied the same recency filters. None of that required an LLM at all. Only two parts actually needed a model: picking a good candidate from the filtered list, and writing the draft itself. Everything in between — fetching, filtering, state tracking — was just ordinary logic dressed up as an AI reasoning task.

So he rebuilt it. Using historical traces from prior runs plus the original instructions, he had a powerful model figure out which steps had become fully deterministic and rewrite those into a Python program. The natural-language skill shrank down to a thin "bootloader" that calls that program, which in turn calls the LLM only twice — for selection and for generation. Token usage dropped 94%. Latency fell 87%. Output quality stayed about the same, and notably, none of this came from switching to a cheaper model.

He frames this as compiling: start loose and flexible in natural language while you're still discovering what a workflow actually looks like, then once it stabilizes, lower the boring parts into code and keep the model only where judgment genuinely matters — deciding what's interesting, writing prose that sounds human. There's an upfront cost to doing that compilation pass, using a big model to study the traces, but it only has to happen once and pays off across hundreds of future runs.

The more pointed argument comes at the end. Model vendors make money selling tokens, so there's little incentive for them to help customers realize most of their agent workflow doesn't need tokens at all. That leaves an opening — for independent builders and tooling — to build the compilers, harnesses, and trace-analysis tools that strip agent workflows down to the parts that actually need intelligence, instead of running a frontier model as an expensive state machine for tasks a for-loop could handle.