Text Generation Inference

Specifications

No specifications recorded yet.

Latest developments

2025

• 28 Mar 🚀 Accelerating LLM Inference with TGI on Intel Gaudi
• 16 Jan Introducing multi-backends (TRT-LLM, vLLM) support for Text Generation Inference

2024

• 1 Feb Hugging Face Text Generation Inference available for AWS Inferentia2

2023

◆ Mistral AI releases Mixtral 8x7B, an open-weights mixture-of-experts language model Model release

• 11 Dec Welcome Mixtral - a SOTA Mixture of Experts on Hugging Face
• 5 Dec AMD + 🤗: Large Language Models Out-of-the-Box Acceleration with AMD GPU

Relationships

Products & technology

• Hugging Face develops this model · 3 sources
• Integrated with TensorRT-LLM · 1 source
• Integrated with vLLM · 1 source
• Integrated with llama.cpp · 1 source
• Integrated with AWS Neuron · 1 source
• Integrated with Google TPU · 1 source
• Integrated with Intel · 1 source
• Amazon SageMaker integrated with this model · 1 source