Google releases TurboQuant vector-compression algorithm, reducing KV cache to 3 bits without loss of accuracy
Google has released TurboQuant, a compression algorithm that quantizes KV cache to 3 bits without training or fine-tuning and without loss of model accuracy, reducing KV-memory use by at least 6×.
Why it matters: Google reports TurboQuant's algorithm details and benchmark results, helping readers assess the usefulness of KV-cache compression for long-context workloads.
