Meet Switchyard: A Rust Proxy and Library That Routes and Translates LLM Traffic Across OpenAI and Anthropic APIs
MarkTechPost Michal Sutter
NVIDIA released Switchyard, a Rust proxy and library that decodes inbound LLM requests, routes them to a selected upstream backend, and translates requests and streaming responses between OpenAI and Anthropic formats. The server exposes routing-overhead timing via switchyard_routing_overhead_ms in Prometheus buckets starting at 0.1 ms. Switchyard is provided as pre-alpha/experimental and explicitly not for production, changing deployments by enabling provider-neutral agent APIs while delaying production adoption until v1.0.
Why it matters
NVIDIA has released Switchyard, an Apache-2.0 Rust proxy and library for LLM traffic. It decodes requests into provider-neutral types, routes them with passthrough, random, LLM-classifier, or stage-router algorithms, and translates responses back into the client's format, so Claude Code or Codex CLI can run against vLLM, NIM, or Ollama unchanged. The project is pre-alpha and not for production. The post Meet Switchyard: A Rust Proxy and Library That Routes and Translates LLM Traffic Across OpenAI and Anthropic APIs appeared first on MarkTechPost.
Related stories
Launch HN: Tokenless (YC S26) – Automatic model switching to save money
usetokenless.com · 1 month ago ·
2
Meta AI Introduces MetaRoCE: A Clean-Sheet RDMA Transport Built for AI-Scale Ethernet
MarkTechPost · 1 week ago ·
32