Olix Computing, a London-based AI hardware startup, raised $312 million in Series C funding led by Netflix co-founder Reed Hastings and Arm Holdings, valuing the company at $3.3 billion. The company is developing the DX-1 chip optimized for LLM inference decode workloads, which it plans to ship in the first half of 2027 as part of an X-1 data center appliance using optical interconnects. Olix's approach stores large KV caches in on-chip SRAM memory rather than slower off-chip HBM, targeting throughput of over 10,000 tokens per second for 100-billion-parameter models.
Baseten's Philip Kiely and Ali Taha discuss inference engineering as a specialized discipline for turning trained model weights into fast, reliable production APIs, covering techniques like quantization, speculative decoding, and cache-aware routing. In one GLM-5.2 experiment, quantizing more of the model increased throughput by 20% while preserving benchmark quality because errors in different layers cancelled each other out. Inference optimization has become as critical as model training itself, enabling gains of 20% to 200% and making models up to 10 times faster at scale.
Open source models like Kimi K3 are reaching frontier capability levels but require massive parameter counts (2.8 trillion), making AMD's MI355X GPU a cost-competitive alternative to NVIDIA's B300 for serving them. On a benchmark with 1,024-token input and 400-token output, the MI355X achieved 952 tokens per second per node at 48 tokens per second per dollar, compared to the B300's 33 tokens per second per dollar, despite some software framework gaps. As AMD ships better day-zero support for these large models and engineers close optimization gaps, NVIDIA's traditional advantage in software maturity becomes less defensible for inference workloads.
Every AI story that matters,
in your inbox by 8am.
TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the
day in two minutes. Follow companies and topics for alerts, or get the
briefing in Slack. Free, no spam, unsubscribe anytime.
Reading TLDRocket needs no cookies, and the readership counts we rely on come from
our own cookieless analytics. Google Analytics is the exception: it sets cookies and
reports to Google, so it stays switched off until you allow it. You can change your
mind any time from “Cookie settings” in the footer.
Strictly necessary
Session security and form protection (tldrocket-session,
XSRF-TOKEN, 2 hours). The site cannot work without them,
so they need no consent.
Always on
Google Analytics 4 (_ga,
_ga_<id>, up to 2 years). Measures which
stories and sections readers use. Google acts as a third-party processor and may
store the data outside the EU. No advertising, no profiling, no data sold.
