Harness engineering has acquired a name and success stories. By OpenAI's own account, it built an internal product with roughly one million lines and no manually written code. Anthropic showed the same model producing a better application after a planner, generator, evaluator, browser testing, and file-based handoffs were placed around it.
But they leave out the line item that matters to the company deploying the agent.
The harness is not a clever prompt written once. It is an operating system around the model. It runs whenever the agent works, and every part has a cost.
The model price buys an attempt. The harness determines whether it becomes an acceptable outcome.
This extends the cost model I developed in Why AI Security Is Becoming a New FinOps Cost Layer and Why Human Oversight Is Becoming a New FinOps Cost Layer, and sharpens the architecture introduced in Mapping the AI Stack.
Harness and security are the two layers of the control plane. Human oversight is a separate plane again — the recurring cost of keeping people capable of stopping the system.
Two Vendors and a Customer
OpenAI and Anthropic build harnesses to make their products more capable. They can justify expensive experiments because the engineering may improve a platform used by many customers.
DoorDash built Flux to delegate engineering work under its own security, reliability, and audit obligations. It reports 130,000 automated engineering tasks in one month and more than 25,000 automated code reviews each week.
Flux uses Firecracker microVMs to isolate execution. Its in-house MCP gateway, called Agent Gateway, controls access to internal systems, scopes permissions, and logs actions. More than 300 reusable workflows are expressed as YAML playbooks that define tools, permissions, validation, outputs, and safety boundaries.
DoorDash reported an average cost of roughly $3 for its AI code reviews in production. A controlled replay on 105 curated pull requests recorded $3.91 per review for its production configuration versus $0.75 for a no-scout baseline.
That fivefold per-review gap is misleading on its own. The production configuration achieved 53.6% weighted recall, compared with 30.7% for the baseline. Cost per real finding was $0.82 versus $0.48 — a gap of roughly 1.7 times. The harness still cost more, but it also caught substantially more real issues. Cost per review and cost per outcome tell different stories.
To a vendor, the harness is product engineering.
To the operator, it is recurring cost.
What Has to Be Priced
For costing, I divide the harness into six operating elements: the execution loop, tools, working memory and state, permissions, evals, and evidence. Permissions here means the harness's connection to credential and approval systems. The separate security layer defines and enforces the boundary around those connections. Evals sit at the overlap.
These elements already exist informally: re-pasted rules, routine approval clicks, and visual judgment of the final answer.
Designing them converts fragile prompting into an operational system with an operating cost.
Evals may be the largest hidden multiplier — and they sit precisely where the harness and security layers meet. The harness runs the evals; the security layer defines what "passed" has to mean.
Bai and colleagues studied 1,030 valid execution traces from seven agent frameworks running GPT-4o-mini across two security-focused datasets, SecurityEval and CVEfixes. Within that bounded setting, they confirmed 170 silent failures: patches that passed syntactic and functional checks yet retained or introduced security weaknesses. Every one of those 170 patches was, by the usual definition, "done."
The failures were not random. They fell into three distinct patterns, and each one defeats functional testing in a different way.
The pattern that should worry an operator most is omission, the largest share: the agent simply leaves out a required control. Nothing looks broken, the failing test now passes, and the missing defense is invisible until something exploits it. Introduction is worse in a different sense — the agent fixes the reported issue while adding a new weakness, so the patch is a net negative on security while reading as a clean success. And the paper found that adding an LLM-based reviewer did not reliably catch these; a second model, cheaply bolted on, is not the same as an eval built to test security properties.
This is why evals are not a one-time gate and not free. Catching these failures requires security-semantic analysis, inspection of every generated artifact rather than only the diff, exploitability checks, and independent review — recurring work that exists precisely because a passing functional test is not a secure outcome. The study does not price that work, and its scope is bounded to one model, Python, and two datasets, so it is not a universal failure rate. But it shows why the eval layer is load-bearing: the machinery required to trust what the agent produced is a cost of its own, and cost per secure successful outcome is where it belongs.
HITL Is Not the Harness
The HITL layer is not the harness, and it is not part of the control plane at all. The harness and security are the two layers of the control plane — the machinery that makes the agent run and the boundary around it. Human oversight is a separate plane: the recurring cost of keeping the people who supervise that machinery capable of stopping it. A fully engineered control plane can still be paired with hollow oversight; they are funded, and they fail, independently.
Evals sit at the harness-security overlap. The harness schedules and executes them; the security layer defines the properties they must test and the evidence they must preserve. Their cost belongs to the control plane. The cost of maintaining capable human intervention belongs to the separate oversight plane.
That distinction matters for allocation. As I argued in the earlier security-cost model, secure operation carries recurring technical controls. The human-oversight model prices reviewer readiness, training, attention, simulations, escalation capacity, and the ability to intervene. Combining those budgets under a vague "governance" label hides which plane is expensive and which one has failed.
What the 20x Result Actually Means
Anthropic compared a solo Opus 4.5 run with a harnessed run using the same model. A planner, generator, and evaluator coordinated through files and testable sprint contracts, while the evaluator drove the application through Playwright.
The solo run took 20 minutes and cost $9. The full harness ran for six hours and cost $200 — over 20 times more expensive — excluding the human work used to develop and tune the approach. Anthropic reported that the solo run's core functionality was broken while the harnessed application worked and contained more functionality.
This primary-source result came from one application, model, and harness. It is a raw run-cost ratio, not a universal harness multiple. It shows that capability improvement can carry a large operating multiplier without changing the model.
The useful question is therefore not whether 20x is too high.
It is: what is our harness multiple for the work we intend to delegate?
Why the Cost Does Not Disappear
Integrations, policies, and evaluation sets can be allocated across future tasks, but the harness does not amortize to zero. Every run still consumes tools, context, isolation, verification, logging, and review.
Subsidized AI is not a subsidized outcome. Credits may reduce the model bill during a pilot, but they do not remove the organization required to operate the system after the credits expire.
Authority makes the permanence clearer. An FAQ assistant may need sampled review. A refund agent needs transaction limits, fraud controls, logs, and escalation. An infrastructure agent needs stronger isolation, rollback, and independent verification.
More authority requires more harness. The business case must include the cost of making autonomy acceptable.
Individual layers are not permanent by default. In a later V2 experiment on a different application, Anthropic reported a $124.70 run using Opus 4.6 and a simpler harness. This was not a same-task before-and-after comparison with the $200 run. The source also found that Opus 4.6 made the evaluator unnecessary overhead for tasks the model could reliably handle alone. Harness assumptions go stale. The recurring obligation is to test each layer and remove it when it no longer improves accepted outcomes.
Measure Your Own Harness Multiple
An operator needs a controlled comparison using its own work.
- Select 20 to 50 representative tasks, including known failure cases and high-authority actions.
- Define acceptance before running anything: functional success, security requirements, policy compliance, and the validation window.
- Run both the unharnessed agent baseline and, where relevant, the current human process. Record model, tool, infrastructure, review, failure, and retry costs.
- Add or remove one layer at a time — state, scoped tools, sandboxing, independent verification, or evidence — and repeat each condition at least three times.
- Report acceptance rate and cost per accepted outcome for every condition. Compare the worst repeat, not only the best.
Harness multiple
= Cost per accepted outcome with the harness
÷ Cost per accepted outcome at baseline
The denominator cannot be self-reported "tasks completed." It must be outcomes that passed the independent acceptance rule. If the baseline produces no accepted outcomes, the ratio is undefined: the harness is an enabler, not a multiplier. Report its cost per accepted outcome and the acceptance-rate change instead.
Then place the result inside the larger FinOps measure:
Cost per secure successful outcome
= AI service costs + harness runtime costs + allocated security-control costs + allocated human-oversight costs
÷ Independently validated secure outcomes
This turns the harness into a decision variable. A layer that raises cost but improves accepted outcomes may be economical. One that adds spend without changing independently measured results may not be.
Stop asking only, "What is the model price per million tokens?"
Ask, "What does the harness cost per secure successful outcome at the authority level we plan to grant?"
The operator cannot borrow that number from OpenAI, Anthropic, or DoorDash. But with the same task set, an acceptance rule, repeated runs, and complete cost allocation, the operator can carry a defensible answer into the room.
References
- DoorDash — "Delegating Engineering Work To Cloud-Based Agents"
- DoorDash — "How DoorDash Built an AI Code Reviewer Engineers Actually Listen To"
- DoorDash — "How We Learned To Trust Our AI Code Reviewer at DoorDash"
- OpenAI — "Harness Engineering: Leveraging Codex in an Agent-First World"
- Anthropic — "Effective Harnesses for Long-Running Agents"
- Anthropic — "Harness Design for Long-Running Application Development"
- Bai et al. — "When Passing Tests Hides Vulnerabilities"