How ChatGPT Optimizes its Agent Loop: Harness, API, and Inference
TLDR ● Covered by 7 sources
OpenAI engineers shared optimization techniques used in ChatGPT and Codex to reduce the cost of agentic AI applications by improving efficiency across three layers: harness, API, and inference. Key techniques include using persistent WebSockets instead of repeated HTTPS connections, sending only delta changes rather than full context on each loop iteration, and managing GPU cache through speculative decoding and KV cache management. These optimizations reduce per-task costs significantly, allowing models like GPT-5.6 Sol to outperform competitors at lower expense.
Why it matters
This article discusses techniques adopted in frontier labs to make API applications more efficient, covering what happens when requests are sent to an AI agent and how the harness layer cuts repeated work.
Also covered by
- The New Stack — Chinese AI competitors may have forced OpenAI’s hand on pricing
- TLDR Dev — How GPT-5.6 fuses frontier intelligence with frontier efficiency
- OpenAI Blog — Advancing the price-performance frontier with GPT-5.6
- The New Stack — OpenAI fixed GPT-5.6 Sol’s most frustrating flaw: Burning limits while it waits
- The New Stack — GPT-5.6 kernel of truth: Sol can cut its own costs, says OpenAI
- OpenAI Blog — How GPT-5.6 fuses frontier intelligence with frontier efficiency
