Meet Switchyard: A Rust Proxy and Library That Routes and Translates LLM Traffic Across OpenAI and Anthropic APIs
MarkTechPost Michal Sutter
NVIDIA released Switchyard, a Rust proxy and library that decodes inbound LLM requests, routes them to a selected upstream backend, and translates requests and streaming responses between OpenAI and Anthropic formats. The server exposes routing-overhead timing via switchyard_routing_overhead_ms in Prometheus buckets starting at 0.1 ms. Switchyard is provided as pre-alpha/experimental and explicitly not for production, changing deployments by enabling provider-neutral agent APIs while delaying production adoption until v1.0.
Why it matters
NVIDIA has released Switchyard, an Apache-2.0 Rust proxy and library for LLM traffic. It decodes requests into provider-neutral types, routes them with passthrough, random, LLM-classifier, or stage-router algorithms, and translates responses back into the client's format, so Claude Code or Codex CLI can run against vLLM, NIM, or Ollama unchanged. The project is pre-alpha and not for production. The post Meet Switchyard: A Rust Proxy and Library That Routes and Translates LLM Traffic Across OpenAI and Anthropic APIs appeared first on MarkTechPost.