Launch HN: Respan (YC W24) – Open-source Rust AI gateway built around plugins
Hacker News Andydy25
Respan launched an open-source Rust AI gateway built as plugins. It’s meant to stop gateway glue code from turning into a monster.
Based on reporting by Hacker News, Andydy25 — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Respan has opened up a Rust-based AI gateway that treats almost everything as a plugin: providers, protocols, modalities, and even features like fallbacks, PII redaction, and smart routing. The company is pitching the design as a small core with public interfaces, so teams can assemble only the pieces they need instead of dragging around a giant all-in-one gateway.
That idea didn’t come out of nowhere. Andy says the team has been building on LiteLLM since 2024 and now serves thousands of orgs. As different customers demanded different API key setups, routes, and limits, the wrapper around LiteLLM kept growing until it crossed 100,000 lines of code. Then a routine LiteLLM upgrade in January caused intermittent connection errors, and the team rolled back. Since then, they’ve stopped upgrading LiteLLM directly and have been backporting fixes by hand.
The new gateway is the answer to that mess. Respan says the core should do little more than dispatch events, with plugins handling the rest. Providers such as OpenAI, Anthropic, and Gemini are all meant to fit into the same model, along with chat, realtime voice, MCP, images, and videos. The company says even quirks get their own tests; one provider implementation, Mistral’s, is just 22 lines long.
There’s also a practical compatibility pitch here. Respan says it can handle tools, images, and reasoning protocols automatically, so an app built for the OpenAI Realtime API can use Gemini Live through the gateway, though not every feature maps cleanly. It also preserves provider-specific fields when the client and provider speak the same API, rather than dropping them on the floor.
The project is open source under the MIT or Apache-2.0 license, and it can run locally if users add their own keys and point a supported client at the new base URL. Respan says the hosted platform is how it makes money, with observability, evals, and unified billing layered on top. The code is on GitHub, it’s still pre-alpha, and the company is openly asking people to build plugins for routing, tools, filtering, observability, and security.
My take — AI-written commentary, not fact-checked reporting
This is what AI infrastructure looks like after the first wave of “just wrap it” optimism wears off: the wrapper becomes the product, then the product becomes a liability. Plugins are the sane answer here, because every team wants one more special case and nobody wants another 100,000-line knot. The amusing part is that the industry keeps rediscovering Nginx, only now with more models and more ways to break things.
Read more about this at: Hacker News
