Google Research Introduces TurboQuant, a Vector Compression Algorithm for Key-Value Cache Optimization
Research publication Provisional 85% confidence first seen
Google Research unveiled TurboQuant, a vector compression algorithm that reduces key-value cache memory footprint in AI models to 3-4 bits without requiring model retraining. The technique achieves 6x reduction in key-value memory size and up to 8x speedup in attention computation on H100 GPUs while maintaining model accuracy.