Fable 5.1 World Modeling: Agent Swarms That Build 3D Cities from a Prompt

Yesterday, a GitHub repo hit the front page of Hacker News with 132 points in under 4 hours. It contains an interactive 3D reconstruction of San Francisco's Union Square — 453 buildings, 129 storefronts, 220 pedestrians, 109 vehicles, cable cars, working traffic lights, day/sunset/night cycles, and two fully explorable interiors (Apple Store and Nintendo SF).

The entire thing — every line of code, every 3D asset, every QA report — was generated by autonomous Claude Fable 5.1 agent swarms from a single written prompt.

I cloned the repo, read every major source file, ran the QA pipeline, and dissected the prompt that produced it. This is what I found.

What Was Actually Built

The repo at PhiloLabs/fable51-worlds ships a pure Three.js web application. No game engine. No proprietary 3D tiles. Every building, sign, tree, and traffic light is generated from open data and public reference imagery by code that lives in the repo. Here's the final tally from the QA report:

Performance at 1920×1080 headless: 51–79 FPS at street level (day), 38–75 FPS (night), with 6.3–8.5M triangles and 1,764–3,522 draw calls. The aerial view drops to 32 FPS — but still navigable.

The Pipeline: How Agents Build a City

The PROMPT.md file in the repo is the seed — a detailed but human-scale brief asking for a "browser-scale urban digital twin." The prompt doesn't hardcode coordinates or specify every building. It defines roles, quality targets, and parallel work streams, then lets the agents figure out the rest.

The pipeline has four stages, and every stage is reproducible from the repo:

graph TD
    A[Single Written Prompt] --> B1[GEO Agent A - Plaza]
    A --> B2[GEO Agent B - Powell St]
    A --> B3[GEO Agent C - Stockton St]
    A --> B4[GEO Agent D - Post St]
    A --> B5[GEO Agent E - Geary St]
    A --> B6[Store Census Agent]
    A --> B7[Apple Store Agent]
    A --> B8[Nintendo Store Agent]
    B1 --> C[BPL Architecture Agent]
    B2 --> C
    B3 --> C
    B4 --> C
    B5 --> C
    B6 --> C
    B7 --> D[BPL Retail Agent]
    B8 --> D
    C --> E[Three.js Runtime Assembly]
    D --> E
    E --> F[Camera-Match QA]
    F --> G[Independent Reviewer Agents]
    G -->|Discrepancies| E

1. Reconnaissance — Parallel Research Swarm

Eight independent agents fan out simultaneously. GEO agents each own a geographic sector — Union Square plaza, Powell Street, Stockton Street, Post Street, Geary Street. They pull OpenStreetMap geometry, USGS elevation data, transit specs, and a per-building storefront census with confidence levels and sources. A dedicated Store Census Agent walks every street edge recording business name, address, category, storefront width, signage, and confidence. Separate Apple Store and Nintendo Store agents study those locations independently.

The prompt is explicit about hallucination: "Do not silently invent uncertain businesses. Mark uncertainty." The final QA report shows 28 unresolved storefronts — all marked with confidence levels — rather than fabricated data.

2. Asset Generation — BPL + Blender as a Library

Five parallel BPL (Blender Python Library) agents generate optimized GLB assets: architecture modules (windows, cornices, doors, columns), street furniture (lamps, signs, benches, bollards), retail fixtures (shelving, counters, display racks), vehicle families (sedans, SUVs, taxis, buses), and vegetation (trees, planters, shrubs). The result is 206 GLB files totaling 5.1 MB — remarkably compact for what it represents.

3. Runtime — Pure Three.js Assembly

The runtime is a TypeScript Three.js application. It reads JSON specs produced by the earlier stages and assembles terrain from elevation samples, builds streets from OSM-derived specs, extrudes building footprints with authored façade specifications, places props and vegetation, spawns pedestrians on a 1,398-node navigation graph, and drives vehicles including Powell Street cable cars. Day/sunset/night transitions are built in.

The code is well-structured: separate modules for terrain, streets, buildings, plaza, collision, player controls (walk, orbit, tour modes), interaction system, and lifecycle management. It's not a toy script — it's a real application architecture.

4. Quality Assurance — Camera-Match Against Real Photos

This is the most surprising part. The QA system uses Playwright to drive the real 3D app, screenshot 34 fixed viewpoints, and diff them against free-licensed photographs taken from the same real-world locations. Four independent reviewer agents (architect, geographer, technical artist, interaction designer) file reports with scores and discrepancy lists.

The scores are honest: geographic accuracy 7.5/10, building recognizability 6.5/10, storefront accuracy 6/10, visual fidelity 6/10. The target was 8–9/10. The gap is documented in detail — 30+ specific discrepancies with FIXED/OPEN status, including winding-order bugs (inward-facing wall quads), clipping plane issues, mirrored textures, and duplicate geometry.

Why This Matters

This isn't a demo of "AI can generate a 3D scene." That's been done. This is a demo of something much more significant: autonomous agent swarms executing a multi-stage production pipeline end-to-end, complete with independent quality assurance, discrepancy tracking, and fix cycles.

The prompt is 1,590 lines — detailed but human-scale. It doesn't micromanage. It defines goals, quality bars, and parallel workstreams, then trusts the agents to figure out implementation. The agents build their own research databases, generate their own assets, write their own runtime code, and audit their own output.

Consider what's not in this pipeline: no human 3D artist, no human QA engineer, no human project manager, no human code reviewer. The prompt is the specification. Everything else is agent labor.

The 6–7.5/10 scores are telling. The output is good — recognizably Union Square — but not production-grade. The details that give a scene soul (accurate material rendering, consistent lighting, proper building proportions) are where the gaps are. These are fixable gaps. The pipeline already iterates on them.

A year ago, building something like this required a team of 3D artists, engineers, and QA staff working for weeks. Today, one person with a clear prompt and 45 minutes of agent compute time can produce a first draft. The quality gap between "agent-generated" and "professional" is shrinking fast.

Bottom Line

Fable 5.1 Worlds is the strongest demonstration I've seen of autonomous AI agent swarms producing a complex, multi-component software artifact from a single specification. It's not perfect — the QA report is refreshingly honest about that — but the architecture is sound, the pipeline is reproducible, and the trajectory is clear. If you want to understand where agentic AI is headed in 2026, clone this repo and read the prompt.