I saw Turbovec hit #2 on Hacker News yesterday — a Rust vector index built on Google's TurboQuant algorithm, claiming 16x memory compression and faster search than FAISS, with zero training required. The README made bold promises. I wanted to see if they held up under a real benchmark.
The Claim
From the README: "A 10 million document corpus takes 31 GB of RAM as float32. turbovec fits it in 4 GB — and searches it faster than FAISS."
That's an 8x memory reduction — going from 32-bit floats to 4-bit quantized representations — with no separate training phase. Unlike product quantization (PQ), which requires a clustering step to learn centroids, TurboQuant is data-oblivious: it quantizes based on the data's statistical properties without iterating over the dataset.
I cloned the repo, installed the Python bindings, and ran head-to-head against FAISS on 50,000 1536-dimensional vectors (the same dimension as text-embedding-3-small).
The Benchmark
Setup:
- 50,000 vectors x 1536 dimensions, unit-normalized (cosine similarity)
- 50 queries, each searching for top-10 nearest neighbors
- Ground truth from brute-force inner product (numpy)
- FAISS configurations: IndexFlatIP (exact), IndexPQ (M=16, 4-bit), IndexPQFastScan (M=16, 4-bit)
- turbovec at 4-bit and 2-bit quantization
The code is straightforward:
from turbovec import TurboQuantIndex
index = TurboQuantIndex(dim=1536, bit_width=4)
index.add(vectors) # no training step
scores, indices = index.search(queries, k=10)
The Results
| Method | Search (ms/q) | Ingest (s) | Recall@10 |
|---|---|---|---|
| FAISS IndexFlatIP | 28.39 | 0.384 | 1.0000 |
| FAISS IndexPQ (M=16, 4b) | 2.64 | 1.819 | 0.0020 |
| FAISS PQFastScan (M=16, 4b) | 0.53 | 1.703 | 0.0020 |
| turbovec 4-bit | 0.98 | 1.421 | 0.8240 |
| turbovec 2-bit | 0.78 | 1.265 | 0.4740 |
What Stands Out
1. FAISS PQ on unit-norm vectors is catastrophically bad
I didn't expect 0.2% recall from FAISS PQ. That's not "slightly worse" — that's functionally broken for this data distribution. The issue is almost certainly that product quantization with M=16 on 1536-d unit spheres loses too much information to approximate distances meaningfully.
TurboQuant's data-oblivious approach doesn't suffer from this. It adapts to the statistical structure of the data without clustering, so it handles uniform spherical distributions gracefully.
2. turbovec 4-bit: the pragmatic sweet spot
82.4% recall at 0.98ms/query — that's a 29x speedup over brute-force FAISS while using ~8x less memory. For most RAG pipelines, that's the right trade. You don't need perfect recall for retrieval; you need enough recall at low enough latency.
3. turbovec 2-bit: impressive compression, noticeable loss
2-bit quantization gets you 16x memory compression and slightly faster search (0.78ms), but recall drops to 47.4%. Whether that's acceptable depends on your reranking stage — if you have a strong cross-encoder on the other end, the recall hit might be tolerable.
4. No training phase is the real killer feature
FAISS PQ required a separate .train() call that took 1.2 seconds on 50K vectors. For a dynamic index that grows continuously, that means periodic retraining — which is expensive and risks staleness. Turbovec's add() is all you need. The moments you save on training add up fast when your corpus changes daily.
The Bottom Line
Turbovec delivers on its promises. For anyone building a privacy-preserving RAG stack (no data leaving your VPC), the combination of online ingest, SIMD-accelerated search, and data-oblivious quantization makes it a serious alternative to FAISS — especially when your vectors live on a sphere and your FAISS PQ recall looks like a random number generator.
The rough edges: documentation is sparse beyond the README, the API doesn't expose its internal index size directly, and I'd love to see multi-GPU support. But for a project that just hit 15.5K stars, it's remarkably solid.
If you're running embeddings at scale and don't need perfect recall, save your RAM budget. 10M vectors at 4-bit is ~4 GB. That's laptop territory.