How enabling two settings tripled our scores on the ARC-AGI-3 benchmark

Buried in a technical note, OpenAI dropped a finding that says more about how we build with these models than about the models themselves. Running GPT-5.6 against ARC-AGI-3, a benchmark designed to test genuine reasoning rather than pattern recall, the team discovered that two overlooked API settings were quietly capping performance. Turn them on, and scores jumped roughly threefold.

The first setting lets the model retain its reasoning across turns instead of discarding the scratch work after each response. Anyone who has watched a chain-of-thought model solve a multi-step puzzle knows the pain of it forgetting its own logic halfway through. Keeping that reasoning intact meant GPT-5.6 could build on prior deductions rather than re-deriving them from scratch every time, which sounds obvious in hindsight but apparently wasn't the default behavior developers were using.

The second lever, compaction, deals with the practical problem of token budgets. Long reasoning chains eat context fast, and without a way to compress or summarize earlier steps, models either truncate their thinking or blow through cost and latency limits. Enabling compaction let GPT-5.6 hold onto the gist of its reasoning without carrying every raw token forward, which OpenAI says improved both accuracy and efficiency at the same time — a combination that rarely shows up together in benchmark tuning.

What makes this notable isn't the raw score jump so much as what it implies about benchmark comparisons generally. If a tripling of results comes from configuration rather than model capability, then a lot of published leaderboard numbers may reflect how carefully someone set up their API call more than any underlying intelligence gap. ARC-AGI-3, built by the same team behind the original ARC challenge, was specifically designed to resist shortcuts like memorization, so seeing it move this much on a settings tweak is a useful reminder that evaluation numbers are only as good as the harness running them.

OpenAI frames this as a lesson for developers integrating GPT-5.6 into agentic workflows: the defaults you inherit from an API are not necessarily the defaults that get you good reasoning. For anyone building products on top of these models, that's a more actionable takeaway than another chart showing a bigger number.