Coalent: The LLM Answer Cache That Swears It Won't Go Stale

Here's the problem every RAG system hits eventually: your agent re-reads the same sources on every call, and the moment a single number changes in a source document, every cached answer about it is silently, invisibly wrong.

Coalent is a Show HN from today (v0.6.2) that claims to solve this with provenance-invalidated semantic caching. It caches the understanding your LLM builds — query-independent atomic claims extracted from the source — keys them by meaning (cosine similarity on embeddings), tracks exactly which source artifacts each unit of understanding depends on, and dirties only the affected units when a source changes.

I cloned it, ran 274 tests, wrote 5 adversarial scenarios against its core invalidation claim, and found: it works. Mostly. Here's where it holds and where it bends.

What It Actually Does

Coalent sits above your retriever. You bring any vector DB, tool, or API as the retriever; Coalent adds a caching layer with three properties that are usually in tension:

• Extractive understanding, not chunks — it caches atomic, source-grounded claims (facts extracted by an LLM), not raw text chunks. One cached unit can answer many different questions.
• Reuse across queries and agents — the cache key is query meaning (embedding cosine), not exact text. Ask "what's the rate limit?" and "how many requests per second?" from different agents and they share the same cached understanding.
• Fresh by provenance — every unit remembers the exact sources it came from. When a source changes, only the units that actually used it go stale. Surgically.

graph TD
  Q1[Query A] --> E[Embedder]
  Q2[Query B] --> E
  E --> SC[Semantic Cache]
  SC -->|Hit| R1[Return cached understanding]
  SC -->|Miss| RET[Retriever]
  RET --> SYN[Synthesizer]
  SYN --> STORE[Store unit + provenance]
  STORE --> R2[Return fresh understanding]
  SRCE[Source change event] --> PROV[Provenance tracker]
  PROV -->|Dirty affected units| SC

What I Tested

I ran the full test suite first — 274 passed, 26 skipped (the skipped ones need API keys for OpenAI/Anthropic providers). Clean bill of health on the code quality side. Then I wrote adversarial scenarios targeting the exact promise: surgical provenance invalidation.

Test 1: Multi-source surgical invalidation

Two sources (HR policy + Engineering practices), two queries, two cached units. Change only the HR doc:

retriever.add("doc:hr", "Leave policy: 21 days annual leave.")
retriever.add("doc:eng", "Engineering uses Rust. Deploys every Tuesday.")

cache = SemanticCache(retriever, StubSynthesizer())
r1 = cache.get("what is our leave policy?")
r2 = cache.get("what language does engineering use?")

cache.source_changed("doc:hr", text="Leave policy: now 25 days.")

r3 = cache.get("what language does engineering use?")
r4 = cache.get("what is our leave policy?")
# r3.cache_hit == True  (engineering unit untouched — GOOD)
# r4.cache_hit == False (HR unit re-materialized — CORRECT)

Result: surgical invalidation confirmed. The engineering unit stayed fresh while the HR unit was dirtied and rebuilt. Exactly one unit dirtied.

Test 2: Shared-source broadcast

Two units derived from the same source doc. Change the doc — both units should go dirty:

retriever.add("shared:spec", "Rate limit: 100 req/min. Auth: OAuth2.")
cache = SemanticCache(retriever, StubSynthesizer())
a = cache.get("what is the rate limit?")
b = cache.get("how does auth work?")

cache.source_changed("shared:spec", text="Rate limit: 500 req/min. Auth: OAuth2.")
both_dirtied = a.unit_id in res.dirtied and b.unit_id in res.dirtied
# both_dirtied == True

Result: both units dirtied. The provenance tracker correctly found every unit referencing that source ID.

Test 3: Content-hash dedup (no-op change)

What if you call source_changed with the exact same text? The result: empty dirtied list. Coalent hashes the content and skips invalidation when the content is identical. This matters because webhook-based change feeds often fire duplicate events.

Where It Bends

The default embedder is HashingEmbedder — a zero-dependency pure-Python implementation that works on character-level similarity (n-gram hashing). It does not understand meaning. This means:

cache.get("how many days of annual leave?")  # builds unit A
cache.get("annual leave days per year for employees?")  # MISSES the cache

The paraphrase misses because the HashingEmbedder sees different token sequences. This is a known, documented limitation — the constructor literally warns you — but it's the default path, so anyone who pip install coalent and starts testing will hit it immediately.

To get the semantic cache behavior (paraphrase hits), you need pip install coalent[openai] and an OPENAI_API_KEY. That's a valid trade-off (zero-dependency core vs. semantic accuracy), but it means the "just works" demo doesn't showcase the main selling point.

Other rough edges I noticed:

• ~50 constructor parameters. The SemanticCache.__init__ has ~50 parameters. Presets help, but the config sprawl suggests the design is still evolving.
• The StubSynthesizer returns placeholder data — {'summary': '[stub] decision-ready understanding for: ...', 'claims': [], 'facts': {}}. This is fine for testing but the quickstart doesn't really demonstrate the "extractive understanding" value prop.
• v0.6.x is very new. The project hit v0.6 on August 15, v0.6.1 on August 17, and v0.6.2 (current) on August 18. The changelog shows rapid iteration — the MCP server was added in 0.6.1, the pool read path in 0.6.0. Things are moving fast.

The Bottom Line

Coalent solves a real problem: RAG caches go stale silently, and the standard fix ("re-embed everything on a timer") is wasteful. The provenance-invalidation mechanism is sound — I verified it surgically dirties only affected units, handles shared sources correctly, and deduplicates no-op change events.

Whether you should use it today depends on your appetite for tracking a pre-1.0 project that's iterating weekly. The architectural idea is strong. The execution is solid for 0.6.2. If you're building a multi-agent RAG system where source documents change independently and caching cost matters, Coalent is worth a serious look — but budget time for the learning curve and expect breaking changes.

The core invalidation claim holds. That's the hard part, and they got it right.