Inference Optimization

132 summarised stories about Inference Optimization, each linking back to the original source. Browse all topics →

Key players

NVIDIA OpenAI Together AI Hugging Face The New Stack MarkTechPost

Monday, 24 August 2026

How XPUs Meet a World-Class AI Factory

NVIDIA 1 month ago 45

NVLink Fusion is presented as a way to integrate custom XPUs into NVIDIA’s existing AI factory infrastructure so scale-up networking, rack-scale architecture, and software can keep up with continuous AI factory operation. The article cites sixth-generation NVLink enabling networking across a 72-XPU domain, with XPU-to-XPU transfer latency 3x lower and packet rate 10x higher than off-the-shelf Ethernet-based alternatives. As a result, XPU builders can reuse mature components to reduce time-to-market and operational risk while scaling unified, semi-custom AI factories that mix CPU/XPU and other accelerators on shared rack infrastructure.

Read the full story

AI Infrastructure GPU Infrastructure AI Efficiency Inference Optimization

With Groq 3 LPX in Full Production, NVIDIA Extends Vera Rubin Inference for Agents

NVIDIA 1 month ago 44 ● 2 sources

NVIDIA announced that its Vera Rubin NVL72-based rack-scale system, NVIDIA Groq 3 LPX, is now in full production and is being adopted by multiple AI infrastructure partners for agentic inference. In an Artificial Analysis benchmark with Gemma 4 31B on 100,000-token contexts, it delivered 3,400 output tokens per second, about 4x faster than the nearest alternative platform. The update changes the inference stack by pairing long-context context processing on Rubin GPUs with LPX’s lower-latency token decode acceleration, and by extending NVIDIA’s Vera Rubin network fabric to enable larger, lower-cost agent workloads at scale.

Read the full story

AI Infrastructure Agentic AI GPU Infrastructure Inference Optimization

Every AI story that matters,
in your inbox by 8am.

TLDRocket reads all relevant sources, removes duplicate coverage, and summarises the day in two minutes. Follow companies and topics for alerts, or get the briefing in Slack. Free, no spam, unsubscribe anytime.