Building an Advanced Agentic Harness

There's a big gap between an agent demo that works once and a system you can actually trust in production. TLDR Dev's new deep dive on building an 'advanced agentic harness' spends a lot of time on that gap, using a deliberately mundane example: a bot that compares cities on population, timezone, and a short written summary. Simple task, but chosen on purpose, because it forces every hard problem an agent framework has to solve.

The piece starts by ripping apart the naive agentic loop everyone builds first: one LLM call, one action, repeat. That's fine for a toy demo, but it falls apart the moment you need nine independent lookups to happen at once instead of one after another. So the fix is to have the planner output a full dependency graph upfront, a DAG, rather than doling out actions one turn at a time. An executor then finds every node that's ready to run, fires them off concurrently through asyncio.gather, and caps concurrency with a semaphore so a fifty-node plan doesn't summon fifty simultaneous API calls and torch your rate limit in one shot.

Tools get the same rigor treatment. Instead of hand-rolled validation for every new tool, each one is defined with a Pydantic model that generates its own JSON schema, matching what Anthropic and OpenAI tool-use APIs expect out of the box. That schema also becomes documentation the planner reads directly, and it lets bad tool calls fail before they ever touch a database or rack up an API bill. Memory gets a similar structural fix: instead of dumping the entire chat history into every prompt, the harness splits memory into working, episodic, and semantic tiers, retrieves only the top-matching pieces using sentence embeddings, and assembles context under a hard character budget rather than letting it balloon indefinitely.

Maybe the most pragmatic idea is the two-tier verification system. Cheap, deterministic checks run first, things like confirming every requested city actually shows up in the final report, and only if those pass does the system spend tokens on an LLM judge to grade subjective quality. A report missing Tokyo fails instantly with an exact reason, zero LLM cost involved. The whole architecture ends up split into three narrow roles, Planner, Worker, and Critic, so no single prompt is trying to plan, execute, write, and grade itself all at once. It's essentially the same idea air forces use for running sorties: the pilot doesn't disappear, they just get wrapped in enough structure that the whole operation stays fast, auditable, and recoverable when something breaks.