Together AI Delivers Top Speeds for DeepSeek-R1-0528 Inference on NVIDIA Blackwell

Together AI · 1 year ago

Together AI launched NVIDIA Blackwell GPU support for its inference platform, achieving 334 tokens/second throughput for DeepSeek-R1-0528 models as of July 17, 2025. The company's optimized inference stack combines custom GPU kernels, a proprietary inference engine, speculative decoding, and lossless quantization to deliver a 32 tokens/second speed improvement over baseline deployments and 2.3x to 2.8x faster performance than prior-generation H200 GPUs. Customers can now access this optimized inference capability through Together's serverless endpoints and dedicated endpoints, with the latter offering up to 84 tokens/second additional speedup for production workloads.