AMD + 🤗: Large Language Models Out-of-the-Box Acceleration with AMD GPU

Hugging Face · 2 years ago · 44

AMD and Hugging Face expanded their partnership to enable large language models to run on AMD Instinct server GPUs without requiring code changes compared to NVIDIA equivalents. An AMD Instinct MI250 GPU with 128GB of memory delivers 2.33x more decode throughput and half the prefill latency of an NVIDIA A100, while also fitting larger workloads that exceed the A100's 80GB capacity. Text Generation Inference is now available in production on AMD Instinct GPUs, with future work planned for AMD Radeon consumer GPUs and Ryzen AI laptop processors.