llm-d and Red Hat announce a partnership
Partnership Provisional 58% confidence first seen
The coverage describes llm-d—an open-source inference framework led by IBM Research, with Red Hat (and Google) involved—demonstrating how it can efficiently serve agentic LLM workloads on enterprise GPUs. In the reported demonstration, the team deployed GLM-5.2 (about 753B parameters, ~39B active) across 544 NVIDIA H100 GPUs, reaching over 6.6 million output tokens per minute at peak with up to 3,000 concurrent coding agents. This matters because it positions the IBM/Red Hat-led effort as enabling enterprises to self-host open models for agentic workloads on existing GPU fleets with improved throughput and potentially lower per-token costs versus commercial APIs.
Decision brief
- What changed
- IBM Research reported that llm-d, an open-source inference framework led with Red Hat and Google, demonstrated enterprise self-hosted agentic LLM inference on 544 NVIDIA H100 GPUs using GLM-5.2. In that demonstration, it reached more than 6.6 million output tokens per minute at peak while supporting up to 3,000 concurrent coding agents with zero preemptions.
- Why it matters
- For leaders evaluating whether to run agentic AI internally, this is a concrete signal that a Red Hat-associated open-source stack is being positioned to increase throughput on existing GPU fleets rather than relying only on commercial model APIs. If the reported efficiency gains hold in production, enterprises could have more leverage over infrastructure cost, deployment control, and model-hosting choices for large-scale agent workflows.
- Evidence
- The claim comes from IBM Research's own write-up about llm-d and its demonstration results, with Red Hat named as a lead partner and Google also involved. The coverage provided here is a single vendor-authored source, so the performance figures and operational implications are not independently verified in the supplied material.
- What remains uncertain
- It is unclear from the provided coverage how reproducible these results are across other models, real enterprise workloads, smaller GPU clusters, or mixed infrastructure environments. The summary suggests lower per-token costs and better use of existing hardware, but the supplied article excerpt does not provide comparative cost data, implementation complexity, or production benchmark details.
- Monitor next
- Watch for independent benchmarks or customer deployments showing llm-d performance, operational reliability, and cost outcomes outside IBM Research's own demonstration.
Analytical support, not advice — assumptions and open questions stated above.