Reasoning Prefills on a Few Open Models, v1.1
Gist
A new test fed open models the first 1% of GPT-5.5 Pro’s reasoning. Qwen jumped the most, which hints it may have learned from GPT-5.5 Pro or something close to it.
Based on reporting by Gist — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
TLDR Dev reran its reasoning-prefill test on a few open models, this time using GPT-5.5 Pro as the teacher. The setup was simple enough: for each problem, each target model got two shots, one normal and one with the first 1% of GPT-5.5 Pro’s reasoning inserted into the model’s reasoning channel. The visible answer was still left to the target model to generate on its own.
The question was whether that nudge would make the model’s answer start sounding more like the teacher’s. To measure that, the post looked at how much of GPT-5.5 Pro’s visible answer showed up in the first 100 tokens of the target model’s answer. The score was the mean of unigram, bigram, and trigram source recall, and the reported deltas are absolute percentage-point changes.
The full evaluation covered 45 problems: 15 STEM, 15 non-STEM, and 15 synthetic puzzles. DeepSeek V4 Flash went from 27.30% to 26.13%, a small drop of 1.17 points. Inkling nudged up from 19.99% to 20.45%, while Kimi K3 rose from 31.11% to 35.65%, a gain of 4.54 points.
Qwen3.8 A95B is the striking one. It jumped from 16.79% without the prefill to 34.97% with it, a rise of 18.18 points. By category, the same model moved from 19.26% to 46.24% on STEM, 20.62% to 33.42% on non-STEM, and 10.49% to 25.23% on the synthetic puzzles. That last bit matters because the puzzles are private, which makes the effect harder to dismiss as simple memorization.
The post’s read is that Qwen may have learned from GPT-5.5 Pro, or from a closely related GPT model, rather than from Opus. Kimi K3 still has the highest overlap with GPT-5.5 Pro in both runs, but its change is much smaller than Qwen’s.
My take — AI-written commentary, not fact-checked reporting
This is another reminder that “model family” matters less than everyone likes to pretend. If a tiny reasoning nudge can drag one model sharply toward a teacher’s style while barely moving another, the real story is training overlap, not magic prompting. The open-model crowd keeps wanting clean lines; the data keep handing out fingerprints instead.
Read more about this at: Gist