AWS is running the AWS Trainium Frontier competition to let teams co-design language-model training setups and optional custom kernels for AWS Trainium, to see what architectures perform best under different hardware constraints. Phase 1 uses a single Trainium2 chip with a 30-minute training budget scored by validation bits-per-byte (val_bpb). The competition is progressing from a single-chip, single-metric test to a four-hour, top-10-team Phase 2 that also adds a 50/50 composite including CORE inference performance.
Meta released Muse Glimmer, a 30B multimodal, open-weights agentic model distilled from Muse Spark and intended for always-on local agent workflows. Meta says its 4-bit quantized version can run in about 24 GB VRAM with around 1.0% average degradation, and that DFlash block speculation yields 3.1x higher throughput on an RTX 5090 (74.9 to 233.4 tok/s). It shifts local agent deployment to single-consumer-GPU and offline setups, replacing cloud calls with self-hosted inference under Apache 2.0 weights.
Efficient Knowledge Distillation for LLMs proposes caching teacher top-K logits offline and using a fused, chunked KL-divergence loss to avoid building full vocabulary-by-sequence probability grids during training. The work reports cutting peak VRAM from about 250GB with dense KL to about 128GB with fused chunked KL, enabling long-context distillation on a single H200-class GPU. As a result, training cost drops enough for more large-scale experiments, and the authors claim near-lossless match to online distillation at 8K context while scaling to 32K–256K contexts with much lower memory.
Startups are targeting limitations in transformer-based large language models as the next step in LLM design. Subquadratic says its sparse attention approach rivals mainstream models on some tasks and it plans to make its SubQ model widely available soon. The push is shifting development toward alternatives like sparse attention, retention-based context handling, smaller hybrid models, and diffusion-style text generation.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
