CodeAgents + Structure: A Better Way to Execute Actions
Hugging Face
Hugging Face found that forcing AI agents to write code inside a strict JSON format beats letting them write free-form code.
Based on reporting by Hugging Face — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
AI agents that write Python code to get things done — so-called CodeAgents — have become a favorite alternative to rigid, JSON-only tool calling because code lets them loop, branch, and chain tool calls in ways plain function-calling never could. The catch has always been parsing: an agent's code lives inside markdown, and markdown is fragile. A missing backtick or a stray second code block and the whole action fails before it even runs.
Hugging Face's new research asks a simple question: what if you force the agent to output its reasoning and its code inside a structured JSON blob, rather than hoping the markdown comes out clean? They tested this across GAIA, MATH, SimpleQA, and Frames, comparing plain JSON tool-calling agents, standard CodeAgents, and a new Structured CodeAgent variant. The structured version won consistently, beating regular CodeAgents by 2 to 7 percentage points on average, with OpenAI models showing the biggest jumps on reasoning-heavy tasks and Claude 3.7 Sonnet standing out too.
The team backed this up by digging through 15,724 agent traces. Only 2.4% had a parsing error on their very first call, but that small failure rate mattered enormously: traces without early parsing errors succeeded 51.3% of the time, versus 42.3% for traces that stumbled out of the gate — a 21.3% gap. Agents that fumble their first action also take longer overall, averaging 4.63 steps to finish instead of 3.18. Forcing an explicit
My take — AI-written commentary, not fact-checked reporting
I like that this is a boring, practical fix rather than another benchmark-chasing model release — reliability wins beat headline wins, and Hugging Face deserves credit for publishing the failure cases too. But the 'structure tax' finding is the real story: it quietly confirms that a lot of agent benchmarks are actually testing model scale and instruction-tuning quality, not clever architecture, so don't expect this trick to rescue your favorite 7B model.
Read more about this at: Hugging Face