Cut coding agent token use with better tool output
The New Stack Prasenjit A. Sarkar
AI coding agents burn tokens before they write code, just by reading tool output. Tighter formats can cut that bill without changing the underlying findings.
Based on reporting by The New Stack, Prasenjit A. Sarkar — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
AI coding agents don’t start at line one. They start by chewing through source files, tickets, build logs, quality reports, and dependency alerts, and that reading shows up on the bill before any code is written. Teams usually chase the obvious levers: model choice, prompt length, request caps. But the format of what tools hand back to the model can matter just as much.
The issue is repetition. When a tool returns a big list of similar records in verbose JSON, the model keeps paying for the same field names, quotation marks, and structural bits over and over. That overhead is useful for many systems. It is not always useful for an agent that just needs to inspect a stack of findings and move on.
That is where TOON, or Token-Oriented Object Notation, comes in. It keeps a schema-like header for uniform arrays, then writes each record as a row. The field names appear once. The values stay intact. It is still a lossless encoding of the JSON data model for the shapes it targets, but the same findings arrive with less repeated scaffolding around them.
The piece’s example is a long issue list, where each entry might carry an identifier, severity, file, line number, status, and more. In standard JSON, those labels repeat for every record. For an agent reviewing 25, 100, or 500 findings, that repetition consumes context that could have gone to code, instructions, or evidence. And that’s the real point: output format is not cosmetic. It is part of the engineering of the workflow.
The right answer is still conditional. JSON remains the better fit for nested or irregular data, and it is still the safer default for many scripts and APIs. But when an LLM needs to consume a large, uniform collection, the article argues for measuring a compact representation against the actual payload and the actual model, not guessing from character counts alone. In one representative 25-issue comparison, TOON used 49% fewer characters than pretty-printed JSON and 33% fewer than minified JSON. That is a useful signal, not a free pass.
The broader lesson is plain: agent costs are partly a context-design problem. If a tool is spraying the same structure back at a model again and again, somebody should be paying attention to the wrapper, not just the model.
My take — AI-written commentary, not fact-checked reporting
Most teams obsess over the model name because it’s the easy thing to brag about. The dull truth is that a lot of AI waste lives in boring plumbing, where repeated field names quietly eat budget like a leak in the wall. Open or closed model, doesn’t matter much if the tool output is doing a little tax fraud on every call.
Read more about this at: The New Stack