Natural-Language Workflows Aren't Software Yet

Every agent platform sells the same promise: write a procedure in plain English, and the agent runs it like a program. "Domain experts write reusable workflows; agents execute them as instructions." It sounds like software. It is not software. That gap is why your multi-step agents keep silently skipping steps, and why a workflow that nails it on Monday falls apart on Tuesday.

"Natural-Language Workflows Are Not Software Yet" (Xu, Guo, Shen, Cheng, Zhang) names the failure and ships a fix: Artic, an artifact-driven workflow compiler. The paper landed in this week's arXiv batch and is the most honest treatment of agent-workflow reliability I've seen in a while.

---

The problem: implicit dependencies

Natural-language workflows carry two structural defects. First, data dependencies are implicit — the executor has to guess which prior result a step should consume. Second, context pressure — under a long or branching instruction, agents lose the plot and follow only part of the procedure. Neither is a prompt-engineering problem. Both are compilation problems.

The method: make the artifacts explicit

Artic transforms a natural-language workflow into an artifact-driven workflow where every step is forced to declare the artifacts it reads and writes, constraints gate what an artifact must satisfy before it's consumed, and control transfers are routed explicitly instead of inferred. That representation exposes the enforcement burden the executor was silently carrying — which lets the compiler spot steps that depend on too much state or contain tangled control logic, and refine them through constrained optimization.

The transformation itself is LLM-assisted, so Artic doesn't just trust it: it decomposes faithfulness checking into local obligations per step and runs scenario-based dry runs to test whether each compiled region actually conforms to the source workflow.

graph TD
  A["Text workflow
(implicit deps, fuzzy control)"] --> B["Artic compiler"]
  B --> C["Artifact-driven workflow"]
  C --> D["Step: declares reads / writes"]
  C --> E["Constraint gates artifact"]
  C --> F["Explicit control transfer"]
  D --> G["Reliable executor"]
  E --> G
  F --> G
  B -. "faithfulness check" .-> H["Local obligations + dry runs"]
  H --> B

The numbers

Evaluated on 488 problem instances across 11 real-world domain workflows:

• +28 percentage points task resolve rate over the original text workflow
• 32 pp more consistent across different models (cross-model)
• 56 pp more consistent across repeated executions with the same model

The consistency numbers are the ones that matter. They're not measuring "smarter" — they're measuring deterministic. The artifact contract converts a procedure that mostly works into one that works the same way every time, regardless of which model you point at it.

Limitations

• The compiler is an LLM, so the check is only as strong as the checkers. Faithfulness is decomposed and dry-run tested, but the transformation can still drift on genuinely ambiguous prose. Artic verifies conformance; it can't manufacture intent the source never had.
• The benchmark is narrow. 488 instances from 11 workflows is real but small, and the workflows skew toward structured, artifact-producing domains (the kind where explicit I/O contracts are natural). A workflow that's genuinely judgment-heavy — "review this and decide" — resists artifact compilation by construction.
• It optimizes reliability, not capability. Artic makes the model follow the procedure. It doesn't make the procedure smarter. If the workflow itself is bad, you now fail consistently.

Why you should care

If you've built agents that run procedures — SOPs, runbooks, data pipelines, multi-tool MCP flows — you've shipped this bug: a step that uses the wrong prior output, a conditional that got skipped under a long context, a "sometimes works" workflow you've learned to babysit. The fix isn't a better model; it's the pattern this paper encodes: declare reads and writes, gate artifacts with constraints, route control explicitly. Prompts degrade under pressure. Artifacts don't.

This is the direction agent engineering is already drifting — from prompting to contracts. A workflow that names its inputs and outputs is testable, auditable, and model-portable. That's not a research nicety; it's the difference between an agent that helps and an agent you can hand to someone else.

---