Turbovec: Google's TurboQuant in Rust — 16x Vector Search Compression With No Training

A 10 million document corpus takes 31 GB of RAM as float32. turbovec fits it in 4 GB — and searches it faster than FAISS. That's the headline. What makes it interesting is how: no training, no parameter tuning, no rebuilds as your corpus grows. Just add() and search.

turbovec hit Hacker News front page yesterday (192 points). It's a Rust vector index with Python bindings implementing Google Research's TurboQuant algorithm — a data-oblivious quantizer that compresses vectors to 2-4 bits per coordinate without ever seeing a training set. I cloned it, built it, ran benchmarks against brute-force search and FAISS, and dug through 4,600 lines of Rust SIMD kernels. Here's what I found.

The Problem It's Solving

Every RAG pipeline faces the same memory wall. A 1536-dimensional embedding (text-embedding-3-large) at 4 bytes per float:

• 100K vectors = 586 MB
• 1M vectors = 5.7 GB
• 10M vectors = 57 GB

FAISS Product Quantization (IndexPQ) cuts this by training a codebook on your data — typically 4-8x compression. But the training step requires a representative sample and a separate train() call. Add new vectors later? Re-train or accept degraded accuracy. This is the standard tradeoff: memory vs. training overhead vs. recall.

Turbovec skips training entirely. It side-steps the tradeoff using a mathematical trick: after a random rotation, every coordinate follows a known distribution, so the optimal quantization buckets can be pre-computed from the math rather than learned from data.

How It Works

graph TD
  V[Input vectors f32] --> N[Normalize to unit vectors]
  N --> R[Random orthogonal rotation]
  R --> L[Lloyd-Max scalar quantization
Precomputed from distribution theory]
  L --> B[Bit-pack: 2-4 bits per coordinate]
  B --> S[Store + length-renormalization scalar]
  
  Q[Query vector] --> QR[Rotate query]
  QR --> K[SIMD search kernel
AVX-512 / NEON / AVX2 / scalar]
  K --> H[Heap insert + renormalize]
  H --> R2[Top-k results]

The pipeline has six steps, and the critical insight is steps 2-3:

1. Normalize. Strip each vector's norm (store as an f32). Now everything lives on the unit hypersphere.
2. Random rotation. Multiply all vectors by the same random orthogonal matrix. After rotation, each coordinate independently follows a Beta distribution converging to N(0, 1/d). This holds for any input data — no training needed.
3. Lloyd-Max quantization. Since the distribution is known analytically, the optimal bucket boundaries and centroids are pre-computed from the math. 4 buckets for 2-bit, 16 for 4-bit.
4. Bit-pack. Each coordinate becomes a 2- or 4-bit integer. A 1536-dim vector goes from 6,144 bytes to 384 bytes (2-bit) — 16x compression.
5. Length-renormalization. One scalar per vector corrects the systematic inner-product bias introduced by quantization. Zero search-time cost.
6. SIMD search. Query rotated once, scored directly against compressed codebooks via hand-written AVX-512/NEON kernels.

What I Tested

I built the Rust library from source, installed the Python bindings, and ran two sets of benchmarks.

# Build
cargo build --release -p turbovec

# Python
source /tmp/tv_venv/bin/activate
pip install turbovec numpy

Synthetic benchmarks (my own)

Random unit vectors at standard embedding dimensions, compared against brute-force numpy dot-product search:

Configuration	TV Search	Brute Force	Speedup	Recall@10	Compressed / Raw
d=128 n=100k 2-bit	14.8 ms	171.8 ms	11.6x	0.493	3 MB / 49 MB
d=384 n=100k 2-bit	32.4 ms	196.0 ms	6.0x	0.513	9 MB / 146 MB
d=1536 n=100k 2-bit	61.3 ms	277.0 ms	4.5x	0.552	37 MB / 586 MB
d=1536 n=100k 4-bit	56.2 ms	379.7 ms	6.8x	0.844	73 MB / 586 MB
d=3072 n=10k 4-bit	22.6 ms	68.5 ms	3.0x	0.874	15 MB / 117 MB

Speedups range from 2.5x to 11.6x, with the biggest gains at lower dimensions where SIMD hits hardest. Recall on random vectors is modest — 49-60% at 2-bit, 84-88% at 4-bit. Random unit vectors are essentially the worst case: no intrinsic structure for quantization to exploit.

Published benchmarks (OpenAI DBpedia, 1M vectors)

The repo includes recall benchmarks on the Qdrant/dbpedia-entities-openai3-text-embedding-3-large dataset. Compared against FAISS IndexPQ at the same bit rate:

Dataset	Bit Width	Metric	TurboQuant	TQ+ (calibrated)	FAISS IndexPQ
OpenAI d=1536	2-bit	Recall@1	0.888	0.901	0.872
		Recall@4	0.999	0.999	0.997
	4-bit	Recall@1	0.967	0.959	0.966
		Recall@4	1.0	1.0	1.0
GloVe d=200	2-bit	Recall@1	0.550	0.572	0.564
		Recall@8	0.909	0.923	0.925

TurboQuant matches or beats FAISS IndexPQ at every measured point, using zero training data. The recall gap between random vectors and real embeddings is dramatic — real data has low-dimensional structure that the quantizer captures naturally.

Speed comparison from the same benchmarks:

Config	TurboQuant	FAISS IndexPQ	Speedup
d=1536 4-bit, single-threaded	0.74 ms/query	2.57 ms/query	3.47x
d=3072 4-bit, single-threaded	1.35 ms/query	5.21 ms/query	3.86x

The Killer Feature: No Training

The data-oblivious design enables deployment patterns no training-based method can match:

• Streaming indexes. Ingest vectors as they arrive. No "collect a batch, train, deploy" cycle. Every vector is indexed at the same quality level as the first.
• Tenant-isolated indexes. In a multi-tenant RAG system, each tenant gets their own index. With PQ, you'd need to train per-tenant codebooks or accept cross-tenant accuracy degradation. Turbovec's codebook is universal — no training, no leakage.
• One-shot deployments. Ship an index on customer hardware without ever seeing their data distribution. The quantizer is optimal regardless of what embeddings they throw at it.
• Incremental persistence. sync(path) persists only what changed — a removal or small append costs milliseconds regardless of index size. Crash-safe at any byte.

The Catch

2-bit recall is application-dependent. On real embeddings with TQ+ calibration, TurboQuant hits 90% recall@1. On GloVe (d=200) or random vectors, it drops to 50-57%. Low-dimensional or weakly-structured embeddings need 4-bit or calibration.

CPU-dependent speed. The SIMD kernels gate on AVX-512BW/AVX2 at runtime. Without them, you fall to a scalar fallback. The speedups I measured are on a Sapphire Rapids Xeon with AVX-512BW — your mileage on an M-series Mac or older x86 will differ.

Dot-product only. The index uses inner-product similarity. L2 distance isn't supported. This covers the dominant ANN use case (cosine similarity on normalized embeddings), but if you need L2, you'll need to pre-process.

Bottom Line

Turbovec is the real deal. The data-oblivious quantization is not a gimmick — it enables deployment patterns no training-based method can match. The SIMD kernels are well-written, the Python API is clean, and the recall/speed numbers on real embeddings beat FAISS at the same bit rate despite FAISS having the advantage of a data-trained codebook.

If you're building RAG at scale and can't stomach the training overhead of PQ, or if you need to index vectors incrementally without rebuilds, this is the library to try. The 16x memory savings at 2-bit (with 89% recall@1 on OpenAI embeddings) means millions of vectors fit in RAM on a single machine. That changes the architecture.

I'll be watching where this goes — especially if the Python ecosystem picks it up as a drop-in FAISS replacement for dynamic indexes.