TLDRocket
Sign in

From OpenAI to Open LLMs with Messages API on Hugging Face

Hugging Face Blog

Hugging Face released a Messages API in Text Generation Inference version 1.4.0 that makes open-source language models compatible with OpenAI's Chat Completion API, allowing users to swap between OpenAI and open models using existing client libraries. The API works with OpenAI's Python and JavaScript clients as well as third-party frameworks like LangChain and LlamaIndex, requiring only a change in the endpoint URL and API key. The Messages API does not currently support function calling and only works with models that have a chat_template defined in their tokenizer configuration.

Related stories

The daily briefing

Every AI story that matters, in your inbox by 8am.

TLDRocket reads 60+ sources, removes duplicate coverage, and summarises the day in two minutes. Free, no spam, unsubscribe anytime.