dk Dark Knight September 11, 2026 6 min read

SynthID-Text Rebuilt From Scratch: AI Text Watermarking, Measured

Since August 2, EU AI Act Article 50 obligates generative AI providers to mark their output as machine-generated — and Anthropic has confirmed every post-deadline Claude embeds a SynthID-Text-based watermark by default, no opt-out. The vendor says it doesn't hurt quality, carries no identifying info, and survives light editing. Nobody can check any of that, because the deployed detectors and keys are closed. A new paper — Watermarks Without Verification (arXiv:2609.09604, Sept 9) — makes exactly this argument. I didn't want to just read it. So I rebuilt SynthID-Text's tournament sampling from scratch in ~100 lines of numpy and ran the paper's core experiments myself. The headline numbers: detection 0.653 vs 0.503 at chance, output distortion equal to changing the sampling seed, and detection dead at 25% token substitution.

How SynthID-Text Tournament Sampling Works

The mechanism is simpler than the marketing suggests. At each generation step you sample 2^d candidate tokens from the model's distribution — I used 1,024 candidates at depth 10. Then you run d knockout rounds. In each round, each token's "g-value" comes from a secret keyed function of (context window, layer, token ID), and the candidate with the higher g wins. The survivor inherits a bias toward high g-values at every layer, so the detector — which knows the key — just averages the g-values of the emitted tokens and compares against 0.5.

# per-token g-value: keyed PRF over (context, layer, token)
def g_value(key, context, layer, token):
    payload = key + struct.pack("<%dI" % len(context), *context[-5:]) \
              + b"|l=%d|t=%d" % (layer, token)
    return blake2b_uniform(payload)  # [0, 1)

# one tournament step: 1024 candidates, 10 knockout rounds
for layer in range(depth):
    gs = [g_value(key, ctx, layer, t) for t in cands]
    winners = gs[0::2] > gs[1::2]
    cands = np.where(winners, cands[0::2], cands[1::2])

My first build had zero signal — and the reason is instructive. I initially keyed g to the pair index inside the tournament, which the detector cannot reconstruct. SynthID-Text only works because g-values are per-token: sampling biases selection toward high-g tokens, and the detector can recompute the same g for any token it sees. Fix that one line and the watermark appears. This is what "unverifiable" looks like in practice: one subtly wrong line, and you silently have no watermark at all — or worse, a false sense of one.

The Results: Seed-Noise Distortion, Glass-Jaw Detection

400-token sequences, 50k-vocab Zipf-ish "LM", depth-10 tournaments:

graph LR
    A[Logits] --> B[Sample 1024 candidates]
    B --> C[10 knockout rounds
keyed by secret g]
    C --> D[Winner token]
    D --> E[Detector: mean g vs 0.5]
    E -->|0.65| F[Watermarked]
    E -->|0.50| G[Clean or edited away]

Why Unverifiability Is the Actual Story

My reimplementation had the luxury of knowing the key. The deployed systems don't offer that. Anthropic says quality is unchanged — my build says that's plausible, since the distortion is seed-level noise. Anthropic says it survives light editing — my build says that's false above ~15–20% token churn. Which is true of Claude? Unknowable. The key is secret, the detector is closed, and no public tool can probe the production endpoint. The paper's authors tested the open-source SynthID-Text implementation on two open-weight models and got the same shape of result I did: prose impact within seed noise, code impact of about three correctness points, detection near chance on code.

That's the governance failure the paper names, and my experiment sharpens it: the technology works exactly as advertised — and it's still unverifiable, because every load-bearing claim (quality, robustness, no hidden identifiers) requires the very key the vendor refuses to publish. A watermark you can't test is a compliance checkbox, not a safeguard. And a watermark that dies at 25% token substitution protects nobody against a motivated adversary anyway. Its only real function is labeling honest, unedited output — which is precisely the output that never needed labeling.

Bottom Line

SynthID-Text is real, cheap, and fragile — three things that are simultaneously true and that the compliance narrative blurs. If you're a builder: assume watermarked text around you, don't count on detecting anything post-edit, and don't count on vendors' robustness claims surviving contact with an editor. If you're a regulator: mandate verifiability, not just marking. An obligation to mark without an obligation to prove is an obligation to say so.

Related