Compile by Training: Local Neural Functions, 83.6% Accuracy

Every recurring text operation — extract the city from this address, normalize this currency string, classify this ticket — sits in the same trap. Either you hand-write brittle rules and regexes, or you pay a remote frontier model on every single call: latency, cost, and a hard dependency on one provider. A new paper from Deng, Nie, and Shieber (the Harvard group behind Program-as-Weights) makes the middle path concrete: compile by training, which turns a natural-language specification into a small, local, versionable neural function. On FuzzyBench-Hard — the subset where their previous fast compiler produced zero exact matches — the compiled functions hit 83.6% semantic accuracy with no teacher model at runtime.

The Problem: Rules Are Brittle, Remote Calls Are Expensive

Recurring text functions are easy to describe and painful to implement. "Extract the departure city and date from this flight confirmation" is one sentence for a human and a small war for a developer — the formats multiply, the edge cases compound, and every new template breaks the regex. The alternative, sending every input to a large remote model, works but accumulates recurring cost and latency on high-volume paths, and quietly bakes a third-party dependency into code that should be yours.

The prior Program-as-Weights compiler attacked this by turning an LLM into a program of reusable prompt-function calls. But on the hard end of its own benchmark — FuzzyBench-Hard — the fast compiler simply came up empty, with no exact matches at all. That's the gap this paper targets.

The Method: A Spec Becomes a Trained Adapter

Compile by training treats compilation as a training run. At compile time, teacher models generate task-specific examples from the natural-language specification. Those examples train a small adapter mounted on a compact interpreter — the compiled artifact. At runtime, the function executes locally with no teacher in the loop, and it can be stored, versioned, and composed like any ordinary piece of software.

flowchart LR
    A[Natural-language spec] --> B[Teacher models generate examples]
    B --> C[Train small adapter on compact interpreter]
    C --> D[Versioned local neural function]
    D --> E[Runtime: local, no teacher, no remote calls]
    E --> F[Compose and chain like ordinary software]

A compiler where the "codegen" step is a few hundred weights on a small interpreter — and the spec, not an implementation, is what you maintain.

The Results: 83.6% Where the Fast Compiler Scored Zero

The headline numbers, straight from the paper:

• 83.6% semantic accuracy on FuzzyBench-Hard — the subset where the Program-as-Weights fast compiler produced no exact matches
• Compile cost: roughly one minute, versus seconds for the fast compiler — the trade for the accuracy jump
• Zero teacher dependency at runtime — compiled functions run without the teachers that generated their training data
• Shipped and demoed: a multi-site website helper, a language-controlled 3D avatar, and a bidirectional English–Claudish translator, all built on compiled functions

The accuracy-per-compile-second ratio is the real result: a ~10x slower compile buys movement from 0% exact matches to 83.6% semantic accuracy on the hardest slice of the benchmark.

Limitations: A Minute of Compile, Semantic Rather Than Exact

The paper's own constraints deserve attention:

• Compile time is real. A minute versus seconds means one-shot, low-volume tasks still favor the fast compiler — this is for functions you'll execute many times
• Semantic accuracy is not exact-match. 83.6% means roughly one in six hard cases still comes back wrong; correctness models and testing are on you
• The benchmark slice is adversarial. FuzzyBench-Hard is precisely where the previous compiler failed outright, so the number is a floor-raise on hard cases rather than a general-purpose win
• Capability inherits the interpreter. The adapter is small, so compiled functions can't exceed what the compact interpreter plus teacher-generated examples can express
• Limited evidence surface. One benchmark subset plus interactive demos — no published error analysis, no stress test of the versioning story across spec drift

Why Builders Should Care: Functions You Own, Not Calls You Rent

For anyone shipping products on top of LLMs, this reframes the economics of the hot path. Extraction, normalization, routing, and formatting functions that run thousands of times a day are exactly the ones worth compiling: pay the minute of compile once, then run indefinitely with near-zero marginal cost, no latency jitter, and no provider in the loop.

The deeper shift is the artifact itself. A compiled function is software — it can be versioned, reviewed, tested, and chained into pipelines the way prompt strings can't. "Ship the spec, train the implementation" inverts the maintainability problem: when a format changes, you edit the natural-language spec and recompile instead of untangling regex history. Program-as-Weights showed you can compile an LLM into a program. This paper shows that on the cases where that fails, training is the compiler that finishes the job.