Stealing Reasoning Traces from Proprietary LLM APIs
Simon Willison’s Weblog 2 months ago 46 ● 3 sources
A paper described how replaying encrypted chain-of-thought (“reasoning”) blocks from proprietary LLM API responses could let attackers recover plaintext hidden reasoning by running the trace across models and then jailbreaking a weaker sibling. The authors said the easiest target was Claude Haiku 4.5, where a removed feature in 4.6 had still allowed an assistant prefix to work. Providers acknowledged the report and the authors could no longer run the same attacks after the fix.