Process-DAG Topology: The Agents That Don't Need to Think So Hard

What problem does this solve?

Clinical trial programming is the nightmare that generative AI promised to fix but secretly can't touch. The task: take a study protocol and produce analysis-ready datasets conforming to CDISC standards (the regulatory data format the FDA expects). It's transformation code with hard constraints — variable derivations, controlled terminologies, traceability requirements — and one wrong column means the submission fails audit.

Jaime Yan put five frontier models through 11 single-shot attempts. None produced a valid subject-level analysis dataset. Not one. LLMs fail on this task not because they can't generate SAS/R code, but because the task demands a specific ordering of transformations governed by regulatory process knowledge that no amount of in-context learning reliably activates. The model doesn't know what it doesn't know, and it doesn't know what order to do it in.

The proposed solution — GxP-Agent — is deceptively simple: encode the regulatory process as a directed acyclic graph (DAG), hand each node to a focused agent with domain-specific skill context and a validation gate, and let the graph topology enforce the ordering that the model can't hold in its head.

graph TD
    subgraph "Process DAG (Simplified)"
        A[Raw SDTM Data] --> B[ADSL Derivation]
        A --> C[ADAE Derivation]
        B --> D[Subject-Level Dataset]
        C --> D
    end
    subgraph "Per-Node Agent"
        E[Worker Agent] --> F[Validate]
        F -->|pass| G[Pass to Next Node]
        F -->|fail| H[Retry with Context]
    end

What's the method?

GxP-Agent decomposes monolithic dataset generation into 15 domain-specific nodes, each executed by a worker agent carrying pharmaverse skill context — the R package ecosystem standard for clinical trial analysis. Each node has:

• Validation gates: the output must match structural expectations before it's passed downstream
• Conditional retry: on failure, the agent retries with the previous error fed back as context
• DAG topology: the graph edges encode the regulatory process ordering that CDISC mandates — which derivations depend on which, what must be derived before what can be joined

The DAG isn't a suggestion. It's a runtime constraint. Agents don't get to reason about what order to do things in — the graph tells them. They focus entirely on execution within their node, not on planning the sequence.

What are the results?

On CDISC-Bench — a new execution-based benchmark built from the FDA's real CDISCPilot01 submission (254 subjects, 49 ground-truth ADSL variables):

• GxP-Agent + Claude Sonnet 4.6: 100% structural match (49/49 variables, 254 correct records) across three independent runs
• Best RAG-augmented baseline: 59.2%
• All single-agent and flat multi-agent approaches: 0%

The DAG topology doesn't just help strong models — it enables weak ones. GPT-4.1 scores 59.2% mean structural match under the GxP-Agent DAG, where it scores 0% under every other architecture. The graph is doing the heavy lifting, not the model.

The approach also generalizes. On ADAE (adverse events dataset; 9-node branching DAG, 55 variables, 1,191 records), GxP-Agent achieved 100% structural match on the first attempt.

What are the limitations?

Three things to keep honest:

1. Domain specificity. The DAG must be hand-authored from regulatory process knowledge. This isn't a zero-shot framework — it requires a domain expert to encode the graph. For well-understood processes (CDISC, HL7, accounting GAAP) this is a one-time investment. For novel workflows, it's ongoing labor.
2. Single benchmark. CDISC-Bench is one FDA pilot study. The ADAE generalization is promising but both operate within the same CDISC family. We don't know how it holds up on other regulatory standards (SDTM, ADaM extensions) or non-regulatory domains.
3. Structural match ≠ clinical correctness. The eval measures whether the right variables exist in the right positions with the right values. It does not verify that those values are clinically meaningful — a column of "5" for every subject would pass structural match as long as the schema is right. This is a first filter, not a final audit.

Why should someone building things care?

The GxP-Agent result is a specific case of a general truth that the agent community keeps rediscovering the hard way: LLMs are terrible sequence planners for constrained multi-step workflows, but they're excellent executors when the sequence is given to them.

The graph topology provides the scaffolding that lets the model focus on what it's good at — natural language to code translation within a narrow, well-defined scope — while the process knowledge lives outside the model, in a structure that can't be hallucinated, can't be skipped, and can be validated independently.

Every agent builder dealing with regulated or safety-critical workflows should read this paper. The insight — encode process knowledge as topology, not as prompting — is portable to drug manufacturing batch records, financial closing procedures, equipment calibration logs, and any domain where "doing things in the right order" is legally non-negotiable. The model proposes; the graph disposes.