Tencent Open-Sources AngelSpec: A Unified Training Framework for MTP and Block-Parallel Speculative Decoding on Hy3 Models

Speculative decoding works by having a small, fast model guess several tokens ahead while the big model checks the guesses in one pass. The trick has always been picking a drafter that guesses well across everything — casual chat, code, math, whatever. Tencent's new answer, in a project called AngelSpec, is to stop pretending one drafter can do that job and instead build two, each tuned for a different kind of traffic.

The reasoning is straightforward once you see the data. Open-ended conversation has too many valid next words for a drafter to guess far ahead reliably, so acceptance rates fall off fast the deeper you predict. Code and math are the opposite: rigid syntax and repeated structure make long stretches genuinely predictable, which rewards a drafter that proposes whole blocks at once. AngelSpec ships a multi-token-prediction (MTP) model trained on broad conversational data for the first case, and a block-parallel model called DFly, tuned on code and math, for the second.

The engineering under the hood does a lot of the actual lifting. The MTP model gets trained with a technique borrowed from EAGLE-3 called Training-Time Test, where each prediction depth is fed the model's own previous guess rather than the correct answer, closing the gap between how it trains and how it's actually used at inference. Freezing the target model's backbone and generating training data directly from the target's own outputs — rather than an external corpus — pushed mean acceptance at temperature zero from 52.8% to 66.4%, with the biggest jumps coming at deeper prediction positions, like GSM8K's third-position acceptance rising from 29% to over 70%.

DFly, meanwhile, builds on an existing architecture called DFlash by adding per-layer weighting on top of the shared context feature, plus a small sequential head that lets later block positions see what was actually accepted earlier instead of guessing blind. On Hy3-A21B, that combination pushes mean accepted length to 4.79 tokens, against 3.69 for plain DFlash and 3.00 for MTP alone — a roughly 60% lift over the MTP baseline it's meant to complement, not replace.

Tencent also open-sourced the plumbing itself: a torch-native pipeline that streams hidden states from live vLLM inference workers over RDMA into training jobs, supports context windows up to 128,000 tokens, and lets six different drafter architectures — including Eagle3 and DSpark for comparison — run through the same config-driven setup. Seven checkpoints are already on Hugging Face and ModelScope, and on the largest model tested, HY3-295B-A21B, the fastest variant hits close to 2.4x the throughput of plain autoregressive decoding.