Google unveils TurboQuant — an algorithm that reduces AI memory usage sixfold

Researchers at Google Research have developed a new memory compression algorithm for artificial intelligence systems called TurboQuant. The technology significantly reduces the amount of RAM used by neural networks during operation, without sacrificing accuracy or performance.

This was reported by TechCrunch, citing the official Google Research blog. The algorithm addresses one of the key challenges of modern large language models—high memory consumption, particularly of the so-called KV cache (key-value cache), which stores intermediate computations so the model does not need to recompute them when generating responses.

How TurboQuant Works

TurboQuant is based on advanced vector quantization—a classical data compression method. It optimizes how keys and values are stored in the cache, eliminating the typical memory overhead associated with standard quantization techniques.

In tests on open models such as Gemma and Mistral, the algorithm compressed the KV cache down to 3 bits per value. At the same time, model accuracy remained on par with the fully uncompressed version, with no loss in quality across tasks such as question answering, code generation, and summarization.

Google reports that the technology reduces memory usage by at least six times. Additionally, on hardware such as the NVIDIA H100, some scenarios showed up to an eightfold speedup in attention computations (attention logits).

Prospects and Limitations

If TurboQuant can be successfully implemented in real-world products, it could significantly lower the cost of running AI systems and allow models to handle longer contexts with the same resources. The algorithm is particularly useful at the inference stage (running trained models), although it does not address the high costs associated with training.

The development is still at the research stage. A full presentation of TurboQuant (along with related algorithms such as PolarQuant and Quantized Johnson–Lindenstrauss) is scheduled for the prestigious ICLR 2026 conference.

Online, the technology has already been jokingly compared to the “compression” breakthrough from the TV show Silicon Valley—specifically Pied Piper—highlighting its ability to achieve extreme compression with almost no loss in quality.

In Brief

Google Research has introduced TurboQuant, an algorithm that uses improved vector quantization to reduce AI working memory (KV cache) by at least six times without losing accuracy. Tests also showed speedups of up to 8×. The technology addresses a key bottleneck in modern models during inference, could lower operational costs, and will be presented at ICLR 2026. It remains a research-stage development, but its potential is considered very high.