The Gap Was Never the Model

The anchor made a claim. A builder just measured it.

The Performance Gap Is an Architecture Gap argued that the distance between a model’s documented capability and what an enterprise actually gets is not a model gap — it is a harness gap. That was an architectural assertion. Now AWS AI Labs has shipped the measurement. Under a single harness they call the Simple Strands Agent, with only narrow per-family interface adapters and no task-specific tuning, they bring every one of twenty-one models to or above its published model-card pass@1 across three agentic benchmarks — and they name the leftover distance plainly: harness friction, not a fundamental limit of a simple single-agent design. The party with the deepest incentive to blame the model instead blamed the seam.

The gap has a name

The AWS contribution is not the leaderboard. It is a construct: the intent-execution gap — the bidirectional mismatch between what the model intends and what the harness executes, and between what the environment did and what is surfaced back. A model means to call one tool; a parsing slip sends another; the harness returns a generic error; the model reads the failure as its own and reasons forward from a false premise. The gap compounds in both directions.

That is the write path, named from the systems side. The intent-execution gap lives exactly where the harness assembles context, authorizes the tool call, and commits state — inside the execution layer, not above it. AWS closes it with structural mechanisms: line-anchored edits that reject ambiguous matches with disambiguating feedback, faithful serialization that preserves the start and end of long outputs so a test failure is never clipped away, malformed calls dropped at the boundary. These are deterministic invariants at the model–tool seam. A governing layer on the read path cannot reach them; it sees the tool call only after the harness has already captured the intent or lost it.

Intent is captured or lost where state is written. The AWS result gives that claim a mechanism and a name — and locates it inside the harness, on the write path, where a ring above it cannot see.

What the trace is for

The second contribution is what you do with the trace once you own it. AWS represents agent runs as paths through a code state-space and measures a solution distance at every step — how far the current repository state sits from a verified-correct fix, and whether the agent moves toward it or backtracks. The finding that matters: at the frontier, pass@1 stops separating models, and the trajectories do not. Two models post nearly identical solve rates along visibly different paths — one descending sharply to the fix, another oscillating through backtracks. The backtrack-edit fraction spans from under one percent to nearly seventeen percent across the field while solve rates stay bunched.

The score cannot see that. The trajectory can. This is the same result the Stanford Meta-Harness ablation reached from the opposite direction: a system searching over harness code reaches roughly fifty-seven best accuracy when its proposer reads the full execution trace, against roughly forty-one on scores alone — and compressed summaries do not recover the gap. Stanford proved trace access is the decisive ingredient. AWS shows what trace access buys once you have it: the diagnosis pass@1 was never going to give you. Assurance is that diagnosis. The substrate it binds to is the trace.

One harness, thin adapters

Note the shape of the AWS solution, because it is the architectural payload. They did not build a separately optimized agent per model, and they did not wrap many harnesses behind a uniform interface. They built one harness, held its execution loop and feedback semantics fixed across every family, and varied only a narrow band of interface adapters — a patch-style editor for one family, atomic edit and view tools for another. Consolidation, with the seam adapted inward.

Hold that against the read-path pattern. “Meeting the model where its training left it” adapts the interface into a single write path that still owns context, authorization, and the trace. It is not the black-box wrapper that governs the seam instead of the write path and swaps the harness mid-loop. Same instinct — accommodate the model’s learned behavior — landing at opposite architectural locations. One consolidates onto a bindable substrate. The other federates away from it.

That other pattern is the heresy this corpus has spent the season naming, and it has a product. Databricks preached the write-path gospel in a book-length manifesto — intelligence and governance built into the system, trace what the model used to answer — then shipped an ambitious new path and named it Omnigent: an open-source meta-harness that wraps Claude Code, Codex, Pi, and the rest as black boxes behind one uniform API and intercepts their tool calls at its server. Each call meets a policy that reads an event — a tool name, its arguments — and a running scalar of what the session accumulated, dollars spent and documents read, and returns allow, ask, or deny.

Give it its due, at its best. In Databricks’ own slow-burn demonstration the policy layer catches a real indirect prompt injection: a poisoned runbook line steers a vendor-review agent to email confidential terms outward, and a session risk score — thirty per document read, denied past fifty — stops the send, live and tamper-resistant. The gate holds. But read the success carefully. The layer read the poisoned runbook, passed it to the model, and kept a number: it retained reads-before-send equals sixty and discarded the instruction that made the send malicious. Tamper-resistance answers whether the verdict can be overturned; it never answers why the verdict was correct. Identity answers who ran the session; the scalar answers whether the action could proceed; the reasoning that connects them — the basis a risk committee has to test — is retained by nothing. It enforces, unarguably, and it does not attest. Databricks named the pitch assurance and shipped coordination.

The AWS paper is what the gospel looks like when someone builds it instead of preaching it. One harness, holding the write path — context, authorization, the trace — meeting each model at the interface inside that harness rather than wrapping many from above. Databricks wraps because the layer beneath it belongs to the providers; AWS consolidates because it still owns the harness. The ring keeps a number; the builder keeps the trajectory — and the trajectory is the only one of the two that can answer why. Same models, opposite architectures, and only the one that owns the write path can be assured, because assurance binds where state is written and the ring sits on the read path by design.

Three roads, one architecture

The strongest signal is what the AWS paper does not do: it cites neither Stanford nor the Berkeley failure taxonomy. Three results, reached independently, converge on one architecture.

Stanford measures information lost under abstraction — reason from summaries instead of traces and lose roughly fifteen points of diagnostic power. Berkeley annotates more than sixteen hundred execution traces across seven multi-agent frameworks and reports that gains over a single coherent agent are often minimal and that failures are structural — generated at coordination boundaries, with verification itself a major locus. AWS adds the third mechanism: intent mistranslated at the model–tool seam, closable by holding execution invariants fixed inside one harness. Information loss, boundary failure, intent mistranslation — three distinct roads, one destination. The trace is the substrate. Consolidation beats federation. The write path is where the architecture is authored.

Three labs, three mechanisms, no citations between them, one conclusion. Independent convergence is worth more than a citation chain — it is what a settled result looks like before the field agrees it is settled.

And the providers build the same way. OpenAI’s Agents SDK turns tracing on by default: every run emits a structured record of model calls, tool calls, handoffs, guardrails, and spans, inspectable in a traces dashboard. Anthropic’s managed-agent architecture separates a durable, interrogable session from the harness that assembles context, so the record of what happened survives independently of the layer that acted on it. Two frontier providers — the two an enterprise most often builds on — treat the execution record as a first-class property of the harness, not a scalar re-derived at the seam. The heresy keeps a number; the field keeps the trace.