Efficient Knowledge Distillation for LLMs proposes caching teacher top-K logits offline and using a fused, chunked KL-divergence loss to avoid building full vocabulary-by-sequence probability grids during training. The work reports cutting peak VRAM from about 250GB with dense KL to about 128GB with fused chunked KL, enabling long-context distillation on a single H200-class GPU. As a result, training cost drops enough for more large-scale experiments, and the authors claim near-lossless match to online distillation at 8K context while scaling to 32K–256K contexts with much lower memory.
ByteDance’s Seed team released SeedRealtime, a native audio-visual full-duplex LLM that processes audio, video, and text in one end-to-end model and supports real-time continuous interaction. It is partially deployable because it is live inside the Doubao app, while ByteDance has provided no technical report, parameter count, or open weights. As a result, external teams can’t integrate the model yet, but Seed is offering a reference architecture shift away from cascaded ASR/VLM/TTS pipelines and external turn-taking components.
Meta released Muse Glimmer, a multimodal 30B open-source model aimed at local agentic use cases like coding and document analysis. It was distilled from Muse to a 30B parameter model and released today under the Apache 2.0 license. Meta also shipped day-0 support in Transformers and llama.cpp (plus vLLM and Inference Endpoints), so developers can load and run it locally with standard tooling.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
