How we really build production-grade AI agents: beyond models, toward data and API quality
Postman Blog
Postman says the reason your AI agent breaks isn't the model, it's your messy APIs and data. Turns out bad schemas beat GPT-5 every time in production.
Based on reporting by Postman Blog — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Postman just published its playbook for why AI agents keep face-planting once they leave the demo sandbox, and the answer isn't the one most teams want to hear. It's not the model. Teams keep swapping in bigger, smarter LLMs and wondering why their agent still hallucinates a support ticket status or double-books a calendar. Postman's argument, drawn from building agent tooling at scale, is that agent failure is overwhelmingly an interface and data problem dressed up as an intelligence problem.
The company breaks agent reliability into three coupled systems: data quality, API quality, and execution quality. An agent doesn't fail because it can't reason — it fails because the API schema it's calling is underspecified, the data coming back is stale or ambiguous, or there's no guardrail to catch a bad action before it ships. Postman calls APIs the new "control plane for intelligence," arguing that if training data taught the model what it knows, APIs and live data determine whether that knowledge is actually true right now. Miss that distinction and even a flawless reasoning chain produces a wrong, confident answer.
The proposed fix is a five-layer "agent reliability stack": data, interface, reasoning, execution, and governance, with most teams pouring resources into the reasoning layer — the model — while starving the other four. Postman points to protocols like Anthropic's Model Context Protocol as an early attempt to turn APIs into structured, machine-interpretable contracts instead of prose documentation an LLM has to guess its way through. Practically, that means canonical schemas, versioned datasets, explicit handling of nulls and edge cases, and rich metadata, not just field names.
Governance gets treated as load-bearing infrastructure rather than a compliance checkbox. Postman insists production agents need audit trails, deterministic replay, scoped access, and human approval gates for high-impact actions, and that skipping this is why companies roll back agent deployments after the first real incident. The company also pushes back on the full-autonomy narrative: the most durable pattern it sees is humans defining intent and constraints, agents executing, systems validating, and humans approving or overriding. Not a stopgap, in their telling, but the actual stable architecture.
The advice for teams building agents right now is almost anti-climactic: stop tuning prompts and picking models, and go fix your API parameter naming instead. Postman's own experience shows a narrow, schema-strict workflow like automated API test generation beating a general-purpose "API assistant" agent, and that cleaning up inconsistent API parameters often moves the needle more than a model upgrade ever does.
My take — AI-written commentary, not fact-checked reporting
This tracks with everything I've seen outside the hype cycle: the model is rarely the bottleneck, your data pipeline and API docs are. Postman has an obvious commercial interest in pushing this story, but that doesn't make it wrong — it's basically the same lesson enterprise software learned twenty years ago, just with a chatbot bolted on front. If your engineers can't use your API from the spec alone, no amount of GPT upgrades saves your agent, full stop.
Read more about this at: Postman Blog