Aurora

Together AI · 3 months ago

Aurora is an open-source reinforcement learning framework that continuously adapts speculative decoding draft models by learning directly from live inference traces without interrupting service. The system achieves a 1.25x additional speedup over static speculators on models like Qwen3 and Llama3, and demonstrates that online training from scratch can outperform carefully pretrained offline baselines. This approach shifts speculative decoding from a static one-time setup to a dynamic self-improving system that reduces infrastructure costs and adapts to changing traffic patterns.