Who Should Be the Human in the Loop?
Human-in-the-loop is a familiar concept in AI Governance.
But another question deserves attention:
Who should that human be?
A reviewer without authority, time or visibility may not provide an effective control. But the question here comes first: how should an organization choose the person for the role?
Under the EU AI Act’s high-risk framework—scheduled from December 2027 for stand-alone systems and August 2028 for product-related systems—providers must design systems so people assigned to oversight can understand capabilities and limitations, monitor operation, remain alert to automation bias, interpret outputs, and override or stop the system where appropriate. Deployers must assign oversight to people with the necessary competence, training, authority and support.
The voluntary NIST AI Risk Management Framework also emphasizes defined oversight roles, training and proficiency.
These frameworks name the attributes. They do not tell an organization how to choose the person.
Building on this, I believe organizations should consider five areas:
• Domain competence — understanding the activity and consequences of error.
• AI and system literacy — understanding what the system can and cannot do.
• Risk and control literacy — recognizing anomalies and escalation triggers.
• Independent judgment — having the knowledge, incentives and position to challenge AI recommendations.
• Intervention capability — knowing how to pause, override, contain, escalate and document.
Agentic AI adds another challenge: the right human may differ across intervention points. One person may set permissions and objectives, another handle escalation, and another have authority to stop or contain the system.
So the governance question is not simply:
“Do we have a human in the loop?”
It is:
“Have we chosen the right human for each intervention point — and can we show why?”
In practice, that means defining the role for each use case, assigning responsibility, training and testing the person, and keeping evidence of why they were fit for the role.
Human oversight is not simply about keeping a person involved.
It is about designing a human role that can govern increasingly autonomous technology.
