Text-Generation Pipeline on Intel® Gaudi® 2 AI Accelerator

Hugging Face · 2 years ago · 13

Intel Habana released a text-generation pipeline for running Llama 2 models on Gaudi 2 accelerators, available through the Optimum Habana library version 1.10.4. The pipeline supports three model sizes (7 billion, 13 billion, and 70 billion parameters) and can generate text from prompts with configurable sampling parameters like temperature and top_p values. Users can integrate the pipeline directly into Python scripts or use it with LangChain for building applications, requiring only Hugging Face account access and agreement to Llama 2's community license.