vLLM
vLLM is an open-source LLM serving engine that has become widely integrated across industry deployments. Recent usage demonstrates its adoption at Netflix for in-house model serving, integration with NVIDIA's reinforcement learning frameworks, support for disaggregated inference on Amazon SageMaker, and incorporation into Hugging Face's job deployment and inference endpoint services, while also seeing active debugging and optimization efforts from organizations like Mistral AI and TRL.
Updated 3 August 2026
Specifications
No specifications recorded yet.
Latest developments
NVIDIA AI Releases Molt: A PyTorch-Native Agentic Reinforcement Learning Framework
MarkTechPost · 1 day ago ·
18
In-House LLM Serving at Netflix
TLDR Dev · 2 weeks ago ·
3
Welcome Inkling by Thinking Machines
Hugging Face Blog · 2 weeks ago ·
40
Disaggregated prefill and decode for LLM inference on SageMaker HyperPod
AWS Machine Learning · 3 weeks ago ·
14
Run a vLLM Server on HF Jobs in One Command
Hugging Face Blog · 1 month ago ·
5
vLLM V0 to V1: Correctness Before Corrections in RL
Hugging Face Blog · 2 months ago ·
45
Heaps do lie: debugging a memory leak in vLLM.
Mistral AI · 6 months ago ·
24
Introducing AutoJudge: Streamlined inference acceleration via automated dataset curation
Together AI · 8 months ago ·
12
August 2026
July 2026
Thinking Machines releases Inkling, an open-source multimodal language model with 975 billion parameters Open source release
- In-House LLM Serving at Netflix
- Welcome Inkling by Thinking Machines
- Disaggregated prefill and decode for LLM inference on SageMaker HyperPod
June 2026
May 2026
January 2026
December 2025
Together AI Integrates PyTorch Reinforcement Learning Capabilities into AI Cloud Platform Partnership
- Introducing AutoJudge: Streamlined inference acceleration via automated dataset curation
- How to run TorchForge reinforcement learning pipelines in the Together AI Native Cloud
June 2025
May 2025
- The Transformers Library: standardizing model definitions
- Blazingly fast whisper transcriptions with Inference Endpoints
April 2025
January 2025
Alibaba releases Qwen2.5 model family including vision-language, extended-context, and mixture-of-experts variants Model release
- Qwen2.5-1M: Deploy Your Own Qwen with Context Length up to 1M Tokens
- Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference
December 2023
Mistral AI releases Mixtral 8x7B, an open-weights mixture-of-experts language model Model release
Relationships
Products & technology
- Integrated with LMCache · 1 source
- Integrated with DeepSpeed · 1 source
- TorchForge integrated with this model · 1 source
- Netflix integrated with this model · 1 source
- AutoJudge integrated with this model · 1 source
- SageMaker HyperPod integrated with this model · 1 source
- Mistral AI deploys this model · 1 source
- Text Generation Inference integrated with this model · 1 source
- TRL integrated with this model · 1 source
- Hugging Face deploys this model · 1 source
- PipelineRL deploys this model · 1 source
- TNG Technology Consulting integrated with this model · 1 source