AL Alfred 2026-08-26 5 min read

96.7% Fabricated — When an LLM Writes Your Life Story

What happens when you ask an LLM to write your autobiography, then check every single line against the documentary record of your actual life?

Heather Renze ran this experiment on herself. The result, published today on arXiv, is the first quantified audit of LLM-generated autobiography against a subject-specific ground-truth corpus. It's brutal reading for anyone who trusts LLMs with personal narrative.

The headline: 96.7% of days had no verified scene.

What problem does this solve?

LLM confabulation is well-studied in factual QA and summarisation, but autobiography is a different beast. It requires sustained narrative coherence across days, months, and years — and the failure modes are subtler than "does Paris exist in France?" An LLM can name your real employer, your real hometown, your real partner — and still invent an entire afternoon that never happened. How do you measure that systematically?

What's the method?

Renze kept a 366-day "page-a-day" book of first-person anecdotal entries, drafted with a conversational LLM. The model received only a template, two exemplar days, and each day's quote — not her personal corpus. Every day was then audited at the anecdote-scene level against an independent verification corpus (emails, calendars, photos, social media, written records) using a four-level rubric fixed before analysis:

She defines the verification-failure rate as the share of days not rated VERIFIED.

What are the results?

mindmap
  root((366 Days Audited))
    VERIFIED
      12 days (3.3%)
    CONTRADICTED
      19 days (5.2%)
    UNVERIFIED / WEAK
      335 days (91.5%)
      Dominant mode: Grounded Drift
        Real people
        Real employers
        Real settings
        Fictional events

The numbers:

Independent re-rating by another coder confirmed the headline result — the original rate was not inflated — though the four-way taxonomy showed only fair-to-moderate inter-rater reliability, particularly at the WEAK/UNVERIFIED boundary.

Renze then regenerated the same days using current named models under the same inputs. 100% verification failure. Grounding the generation in the subject's corpus (providing the LLM with the actual life record) significantly improved things — but still left 83.3% residual failure.

What are the limitations?

This is a single-subject case study — n=1 by design. The author and subject are the same person, which creates familiarity that could affect auditing. The four-level rubric's boundary between WEAK and UNVERIFIED was found to be unreliable, which the author notes transparently. The verification corpus, while extensive, is inevitably incomplete — absence of evidence is not always evidence of absence. And the LLM used for the primary experiment is not named (it was the model available at the time through a specific conversational interface), though the regeneration with current named models confirms the pattern holds.

Why should someone building things care?

Because LLMs are being deployed in contexts where personal narrative matters: biography generation, memorial writing, therapy tools, journaling assistants, patient history summarisation. This paper puts a number on something we've all suspected: LLMs do not write true stories about people, even when they know those people.

The dominant failure mode — grounded drift — is the most dangerous because it's the hardest to detect. A paragraph about your childhood that names your real school, your real teacher, and a field trip that never happened passes the smell test. It feels true. That's precisely why it's insidious.

The paper also contributes a reusable audit instrument and demonstrates that even grounding in the subject's actual corpus leaves 83.3% residual failure. We are not close to trustworthy autobiographical generation. Anyone building a product that asks an LLM to narrate a person's life should read this paper — and plan their evaluation strategy accordingly.

The code and derived data are available at github.com/heathriel/synthetic-memoir-audit.

Related reading