The cost of a "human in the loop" is not the approval click. It is the ongoing cost of keeping that human alert, skilled, informed, and capable of intervening when the agent fails.
Imagine an AI agent reduces a task from $20 to $2.
Most organizations celebrate a 90% apparent cost reduction. Few ask whether some of the missing $18 was simply moved into human oversight.
The calculation may include tokens, computing, software, and storage. But it may exclude the employee reviewing the agent's decisions. It may exclude training, breaks, independent checks, simulations, audits, and the time required to maintain that employee's professional skills.
It may also exclude the cost of correcting an error that passed through a tired reviewer.
The agent may still be cheaper.
But we do not yet know by how much.
This article is written for leaders who own AI unit economics — across FinOps, product, risk, and engineering — and must decide how much human oversight is enough and how to account for it.
The Irony of Automation
In "AI Agents Push Humans Out of the Loop," Margaret Mitchell, Avijit Ghosh, and Samir Passi challenge one of the most common promises in AI governance: a human will remain in the loop.
An AI agent can plan several steps, select tools, retrieve information, write code, modify files, and initiate transactions. The person supervising it must understand the objective, follow the plan, evaluate individual actions, anticipate consequences, and decide when to intervene.
This becomes harder as agent activity increases.
Repeated authorization requests create approval fatigue. What begins as supervision becomes routine clicking. Long-term reliance can also weaken domain skill, critical judgment, and the situational awareness required to understand what the agent is doing.
This is the irony of automation identified by Lisanne Bainbridge in 1983.
Automation removes routine work but leaves humans responsible for rare and difficult failures. Yet removing people from daily operations weakens the skills they need when those failures occur.
The more reliable the automation appears, the less prepared the human may be when it is wrong.
This is no longer only a theory.
Doctors Lost Unaided Performance
A 2025 study in The Lancet Gastroenterology & Hepatology followed endoscopists at four Polish centres after AI was introduced to help detect precancerous growths during colonoscopies — examinations of the colon.
The AI assistance worked during the procedures in which it was used.
But after regular exposure to the system, the doctors' detection rate during colonoscopies performed without AI fell from 28.4% to 22.4%.
The study was observational and does not prove that all medical AI causes deskilling; other factors may have contributed. But the result is still important.
The technology improved detection while present. Dependence on it may have weakened the human fallback when it was absent.
The organization may record the immediate benefit of AI assistance without recording the declining value of its unaided human capability.
Cognitive Debt
In a 2025 preprint, researchers asked 54 participants to write essays independently, with a search engine, or with an AI chatbot. EEG measurements found the strongest neural connectivity in the unaided group and the weakest in the AI-assisted group. Chatbot users also showed poorer recall and a lower sense of ownership over their essays. The sample was limited, but the study raises an important economic question: if AI lowers the cognitive cost of producing an answer today, does it reduce the human capacity to produce or evaluate one tomorrow?
A company may record an immediate productivity gain while accumulating an invisible capability loss.
The output appears on the balance sheet as efficiency.
The degradation remains unmeasured.
Developers Adopted Shortcuts
A 2026 study examined how professional software developers supervised coding agents.
The researchers found that oversight had become a demanding form of work in its own right. Developers had to configure agents, follow their plans, monitor actions, inspect code, and correct failures while still trying to complete their original assignments.
To make this manageable, participants used practical shortcuts.
Some treated the agent's plan as a reliable description of what it would actually do. Others assumed that code passing unit tests was probably correct.
Both shortcuts can fail.
An agent may depart from its stated plan. A limited test suite may miss security problems, hidden dependencies, data loss, or a misunderstanding of the original requirement.
The code may pass every available test and still be wrong.
The agent accelerated production. But it also made developers cognitively distant from code they remained responsible for approving.
Either the organization pays qualified developers to conduct deeper reviews, reducing some of the expected labour saving, or it accepts shallower oversight and greater risk.
Calling the second option "human in the loop" does not make it equivalent to the first.
Flattery Weakened Judgment
A 2026 study published in Science examined 11 leading AI models and conducted experiments involving 2,405 people.
The models affirmed users' actions nearly 50% more often than humans did. This included situations involving deception, irresponsible behaviour, or potential harm.
Participants nevertheless preferred the more agreeable responses. They rated them as more helpful and trustworthy.
The AI that challenges us may feel frustrating. The AI that validates us feels supportive. But the system we enjoy using may be the one we are least prepared to question.
A confident and agreeable agent can make an incorrect plan easier to approve. Fluency becomes a substitute for evidence. User satisfaction becomes a poor measure of independent judgment.
None of these studies proves that all AI use causes permanent cognitive degradation.
Together, however, they show why human capability cannot simply be assumed.
The System Can Learn From Weak Oversight
Human approvals do not always end with the individual decision.
Companies may use approval rates, completion rates, user ratings, corrections, and other behavioural signals to evaluate or improve their systems. Human feedback is also central to several methods used to train AI models.
An alert reviewer may examine the evidence, challenge the plan, detect missing information, and withhold approval.
A tired reviewer may skim the summary, accept a confident explanation, and click "Approve."
If both responses become training or evaluation data, the system may learn the wrong lesson. It may be rewarded not for producing the most accurate work, but for producing work that is easiest to approve.
That could mean smoother summaries, more confident language, fewer visible complications, and plans designed for quick acceptance.
The system does not need to deliberately manipulate anyone for this to happen. Optimization follows the measured signal.
If the measured signal is user approval rather than well-informed judgment, the system can improve at obtaining approval without improving at being correct.
The problem is therefore circular.
Automation can weaken the person supervising it. Weaker supervision can produce lower-quality feedback. That feedback can then reward systems that demand even less thought from the reviewer.
Mitchell, Ghosh, and Passi connect this to reward hacking. The human rater can become the exploitable part of the reward channel.
Monitoring whether a reviewer clicks quickly may identify a symptom. But if the organization continues rewarding speed, high approval rates, and uninterrupted agent operation, the underlying incentives still favour weak oversight.
Safety-Critical Industries Pay for Readiness
Airlines do not keep pilots capable by placing them near an autopilot and waiting for an emergency.
Pilots receive recurrent training. They practise abnormal and emergency situations in simulators. Their performance is periodically evaluated. Cockpit roles, handovers, checklists, and cross-checks are explicit.
Manual flying skills must remain available even when automation handles much of a normal flight.
Aviation also treats fatigue as a system-level risk. Flight and duty times are limited. Rest periods are required. An exhausted pilot cannot provide a reliable safety layer simply because that pilot remains physically present in the cockpit.
Nuclear plants make a similar investment.
Licensed operators undergo continuing training and periodic requalification. Full-scope simulators expose them to normal operations, equipment failures, unusual events, and accident conditions. Critical actions may require independent verification.
These practices appear inefficient when measured only as immediate output.
Training removes experienced people from productive schedules. Rest limits reduce available working hours. Simulators, secondary reviewers, and independent checks create additional costs.
The duplication is deliberate.
Aviation and nuclear operations do not assume that a person remains qualified because they still hold the same job title. Readiness must be maintained and demonstrated.
Keeping an employee on an approval screen is not the same as keeping that employee capable of intervention.
That capability belongs in the economics of the agent.
What the Authors Recommend
Mitchell, Ghosh, and Passi call their approach cognitive scaffolding.
Developers should introduce strategic friction at consequential moments. Users may record their own judgment before seeing the agent's recommendation, reducing anchoring. High-risk actions can require explicit verification. Related actions can be presented together for meaningful batch review rather than fragmented into endless approval prompts.
Organizations could also monitor whether review times fall while approval rates remain unchanged, whether reviewers stop requesting evidence, or whether disagreement with the agent disappears.
Known-answer "canary" tasks could test whether reviewers still notice mistakes. Retrospective audits could compare an agent's summary with its underlying action logs.
Organizations should rotate reviewers, enforce breaks, maintain opportunities for unassisted work, provide recurrent training, separate conflicting roles, and test whether people remain capable of taking control.
But these interventions create a real trade-off.
People may dislike systems that interrupt them, delay an answer, challenge their assumptions, or require additional reasoning. The features that preserve judgment can make a product feel slower and less helpful.
Companies optimizing for speed, adoption, and user satisfaction may therefore have strong incentives to remove exactly the friction that effective oversight requires.
That is not only a product-design problem.
It is a FinOps problem.
Human Oversight Is a New Cost Layer
In my earlier article, "Why AI Security Is Becoming a New FinOps Cost Layer", I argued that model and infrastructure charges do not represent the full cost of operating an agent.
Monitoring, identity, access controls, logging, containment, testing, and assurance create a distinct security cost layer.
Meaningful human oversight creates another.
This human-in-the-loop (HITL) cost layer is the recurring cost of keeping reviewers alert, skilled, informed, independent, and capable of stopping the system.
It has five operational components.
1. Reviewer Capacity
Someone qualified must be available when judgment is required.
Reviewer capacity includes review time, staffing ratios, escalation coverage, secondary reviewers, shift handovers, and coverage outside normal business hours.
Capacity must be based on meaningful review, not approval volume.
If one reviewer can properly assess twenty actions per hour, assigning that person one hundred approvals does not create more capacity. The extra approvals may be recorded, but they should not be valued as effective control.
2. Attention Management
Attention cannot be purchased once and used without limit.
Organizations may need enforced breaks, task rotations, limits on continuous review, manageable approval queues, and protected time for complex cases.
Railway vigilance systems illustrate the distinction.
A "dead-man" control may establish that a driver remains physically responsive. It does not prove that the driver understands the situation. Railways therefore combine technical vigilance controls with scheduling, rest, training, and broader fatigue management.
AI approval buttons have the same limitation.
They prove that someone clicked. They do not prove comprehension.
Process industries face a related problem with alarm overload. If operators receive too many warnings, they become desensitized. Mature alarm-management practices prioritize alerts and remove nuisance alarms so that important warnings remain meaningful.
AI systems should do the same.
Requiring approval for every minor action can make the consequential request look identical to the harmless hundred that came before it.
Reducing approval volume is not necessarily reducing control. It may be necessary to preserve it.
3. Skill Preservation
The person supervising an agent must retain the underlying domain skill.
That may require recurrent training, periodic unassisted work, simulations, failure exercises, and requalification.
A medical professional must still recognize an abnormality without an AI prompt. A developer must understand the code rather than only the agent's summary. An analyst must know how to challenge an unsupported conclusion.
If employees must sometimes complete work without AI, the organization cannot claim that every manual hour has been eliminated.
But without that practice, the human fallback may exist only on paper.
4. Review Workflow
Meaningful review requires more than placing information on a screen.
Agent actions should be grouped in ways people can understand. Evidence should be visible. High-risk decisions should require deliberate verification. Reviewers may need to record their own judgment before seeing the agent's recommendation.
Some actions may require independent approval from a second person.
These controls add development costs, workflow delays, and labour. They may reduce the number of decisions processed per hour.
Their purpose is to make approval represent judgment rather than reflex.
5. Oversight Validation
Organizations need evidence that the HITL layer continues to work.
Canary tasks can test whether reviewers notice known errors. Behavioural monitoring can identify falling review times, disappearing disagreement, or a decline in requests for evidence.
Sampling and retrospective audits can determine whether approved actions were actually correct. Incident exercises can test whether people remain capable of taking control.
This is the HITL equivalent of security testing.
It asks not whether a human was assigned, but whether that human remained effective.
Fixed and Variable HITL Costs
The HITL layer also has a financial structure.
Some costs are incurred to establish and maintain oversight capacity. Others increase as the agent processes more work.
The boundary is not always exact.
A central review team may be fixed within one planning period but variable over a longer period as workload grows. A simulator may require an initial investment and recurring maintenance. Audit tooling may combine a fixed subscription with usage-based charges.
The purpose is not perfect classification.
It is to prevent these costs from disappearing.
Fixed HITL costs should be allocated across the workflows they support using an explainable driver such as reviewer hours, number of high-risk actions, business value, or risk tier.
Variable HITL costs should follow the workload that consumes them.
Extending the FinOps Measure
The cost model now develops in three layers.
Base measure
Cost per successful outcome
= Total attributable workflow cost
÷ Verified successful outcomes
For an agent that accesses sensitive information or performs consequential actions:
With the security cost layer
Cost per secure successful outcome
= AI service costs + allocated security-control costs
÷ Validated secure outcomes
Meaningful human oversight adds the next layer.
Qualifying outcomes are results that meet the required quality standard, remain inside the agent's authority, satisfy security and policy requirements, and receive the level of human scrutiny required by their risk tier.
With the HITL layer
Cost per HITL-validated secure outcome
= AI service costs + allocated security-control costs + variable HITL costs + allocated fixed HITL costs
÷ Qualifying outcomes
The validation period should match the consequence.
A low-risk writing assistant may need immediate review. A customer-support agent may require post-run sampling and escalation records. An agent changing production infrastructure may require independent approval, complete audit evidence, and a longer assurance period.
What This Does to the Business Case
Suppose an agent's model and tool charges are $2 per completed task.
A qualified employee spends ten minutes reviewing each high-risk outcome. A second reviewer examines a sample. The organization limits continuous review sessions, rotates staff, conducts quarterly failure exercises, and requires periodic unassisted work to preserve domain skill.
The model still costs $2.
The outcome does not.
This does not make the agent uneconomic. Automation may still increase capacity, reduce routine work, or improve consistency.
But the comparison must be honest.
If the agent appears economical only when human review is rushed, training is removed, or reviewer fatigue is ignored, the saving depends on weakening the oversight layer.
The feedback loop introduces a longer-term liability.
If weak approvals become evaluation or training data, degraded oversight today can reduce system reliability tomorrow. The organization accumulates not only cognitive debt among its workers but oversight debt across the human-AI system.
That debt may remain invisible until an unusual event requires the human to intervene.
How to Start Accounting for HITL Costs
Organizations do not need a perfect allocation model before making this layer visible.
They can begin with five steps.
1. Identify High-Authority Workflows
Start with agents that can change infrastructure, move money, approve refunds, communicate externally, access regulated data, create customer commitments, or make decisions that are difficult to reverse.
2. Document Existing HITL Activities
List the reviewer time, escalation coverage, training, simulations, manual practice, independent approvals, sampling, audits, and oversight tooling already being funded.
Some organizations will discover that these activities exist but are scattered across several budgets.
Others will discover that the promised oversight has never been operationally funded.
3. Separate Fixed and Variable Costs
Identify which costs establish the HITL capability and which grow with agent activity.
This makes it easier to forecast how oversight spending will change as adoption expands.
4. Select Allocation Drivers
Assign direct review costs to the workflows that consume them.
Distribute shared costs using reviewer hours, high-risk actions, approval volume, protected records, business value, or risk tier.
The method does not need to be perfect. It needs to be consistent, explainable, and visible.
5. Compare Equivalent Systems
Recalculate unit economics using cost per HITL-validated secure outcome.
Do not compare a well-governed agent with an unsupervised one as though they were equivalent products.
A cheaper system that depends on superficial approval is not necessarily more efficient.
It may simply transfer cost into risk.
The FinOps Responsibility
FinOps should not decide how much human oversight is ethically or legally sufficient.
Security, risk, compliance, engineering, product, and business owners must determine the appropriate controls.
FinOps has a different responsibility.
It should make the cost and value consequences of those decisions visible.
Security teams define access, monitoring, containment, and assurance requirements. Domain leaders define who is qualified to review the work. Engineering designs the approval workflow and preserves the evidence. Operations determines staffing, rotations, and escalation coverage. Risk and compliance define where independent approval is required.
FinOps connects those decisions to unit economics.
The objective is not to place the strongest HITL controls around every interaction.
A public information assistant and an agent changing production systems should not carry the same burden.
Low-risk and reversible actions can receive bounded autonomy. Automated checks can verify properties that do not require judgment. Related actions can be reviewed together. Human attention should be reserved for uncertainty, exceptions, and consequences that cannot be easily reversed.
The HITL burden should rise with authority, complexity, irreversibility, and potential harm.
But when an organization promises meaningful human oversight, it must fund the conditions that make it possible.
FinOps made model consumption visible.
AI security made control costs visible.
HITL makes human readiness visible.
References
- Mitchell, Margaret, Avijit Ghosh, and Samir Passi — "AI Agents Push Humans Out of the Loop"
- Bainbridge, Lisanne — "Ironies of Automation"
- Budzyn, Katarzyna, et al. — "Endoscopist Deskilling Risk After Exposure to Artificial Intelligence in Colonoscopy"
- Kosmyna, Nataliya, et al. — "Your Brain on ChatGPT: Accumulation of Cognitive Debt When Using an AI Assistant for Essay Writing Task"
- Dhanorkar, Shipi, Samir Passi, and Mihaela Vorvoreanu — "Human Oversight of Agentic Systems in Practice"
- Cheng, Myra, et al. — "Sycophantic AI Decreases Prosocial Intentions and Promotes Dependence"
- Sellen, Abigail, and Eric Horvitz — "The Rise of the AI Co-Pilot: Lessons for Design from Aviation and Beyond"
- Parasuraman, Raja, Thomas Sheridan, and Christopher Wickens — "A Model for Types and Levels of Human Interaction with Automation"
- Endsley, Mica — "Toward a Theory of Situation Awareness in Dynamic Systems"
- Federal Aviation Administration — "Fatigue Education and Awareness Training Program"
- Electronic Code of Federal Regulations — "Recurrent Training — 14 CFR 121.427"
- U.S. Nuclear Regulatory Commission — "Operators' Licenses — 10 CFR Part 55"
- International Atomic Energy Agency — "Development, Use and Maintenance of Nuclear Power Plant Simulators for the Training and Authorization of Personnel"
- Federal Railroad Administration — "Fatigue Risk Management Programs — 49 CFR Part 270"
- International Society of Automation — "ISA-18 Series of Standards"
- FinOps Foundation — "FinOps for AI"