A/B test models in production

Together AI · 1 week ago · 50

Together AI's platform enables A/B testing of LLM models at the endpoint level, allowing teams to split live traffic between a control and up to 20 variant deployments with configurable percentages. The system uses fixed traffic shares that remain constant during autoscaling, with routing logic handled server-side rather than in application code. Testing can progress from initial 95/5 splits through ramping stages to full 50/50 comparisons, then winners are promoted via blue-green rollout and experiments deleted without residual cleanup.