llm-chat-completions-server 0.1a0

Simon Willison has a habit of building small, sharp tools that quietly solve annoying problems, and llm-chat-completions-server is the latest one. It grew directly out of work on LLM 0.32rc1, which introduced content-addressable logs — a fancy way of saying the tool now hashes individual pieces of a conversation instead of storing the whole thing over and over again every time a chat gets longer.

That matters because of how the OpenAI Chat Completions API actually works. The client, not the server, is responsible for tracking conversation state. Every new request has to resend the entire message history, so a ten-turn conversation means the tenth API call is hauling around nine previous turns just to ask one more question. Willison's new schema design uses hashes of each message part so the same content doesn't get duplicated on disk every time.

To actually test whether the de-duplication logic held up, he built a small server plugin rather than just writing unit tests. Install LLM's prerelease with uv, add the plugin, and running llm chat-completions-server -p 9001 spins up a local server on port 9001 that speaks the Chat Completions dialect. Point any OpenAI-compatible client at it and it'll happily route requests to whatever models you've got installed through LLM's plugin ecosystem — Qwen, or anything else you've wired up.

The detail Willison seems most amused by is who wrote the code. He credits a model he calls GPT-5.6 Sol with authoring the whole plugin, noting it apparently has the OpenAI Chat Completions request and response shape memorized cold. Given how many tools now exist purely to mimic that API's format, that's not a huge surprise — but it's a neat bit of self-referential irony, an AI model writing the server that lets other AI models pretend to be that exact API.