OpenAI Blog
·
1 month ago
Braintrust engineers use OpenAI's Codex with GPT-5.5 to automatically convert customer requests into code for faster experimentation and development. The system processes customer-defined requirements and transforms them directly into executable code, reducing the manual coding workload for engineers. This enables Braintrust's team to run more experiments and deploy features more quickly than traditional development workflows.
.png)
Together AI
·
1 month ago
Together AI optimized its speech-to-text serving stack to transcribe 20 hours of audio in under 10 seconds using NVIDIA's Parakeet-TDT 0.6B v3 model. Key improvements included multi-profile TensorRT compilation for variable audio lengths, moving decoder branch logic onto GPU with conditional CUDA graphs for 2-3x faster decoding, eliminating redundant CPU copies through shared memory and Unix domain sockets, using epoll for streaming connection handling, and calling gc.freeze() to prevent garbage collection pauses. The changes address the full data path from preprocessing through GPU inference, reducing latency and tail latency spikes in production voice agent systems.
Hugging Face Blog
·
1 month ago
PyTorch's torch.profiler module provides tools to analyze performance bottlenecks by exporting profiler tables showing execution time statistics and profiler traces showing temporal operation sequences across CPU and GPU. Running a 64×64 matrix multiplication and addition operation showed the GPU idle 99% of the time due to launch overhead, but increasing to 4096×4096 matrices shifted the workload from overhead-bound to compute-bound, with GPU kernel execution time rising from 23 microseconds to 4.495 milliseconds. Understanding these profiling artifacts—tables for identifying hotspots and traces for investigating kernel dispatch chains and CPU-GPU synchronization gaps—enables developers to systematically identify and address performance constraints in PyTorch models.
