"Keep a human in the loop" is the most repeated and least specified piece of AI advice. A human in which loop, for which decisions? Put a person in front of every action and you have just rebuilt the slow manual process the AI was meant to replace — and worse, you have created rubber-stamp fatigue, where the human clicks approve on a thousand trivial items and is too numb to catch the one that matters. The real question is not whether to keep a human in the loop, but where to place the gate.

The three axes that decide

In Agent of Record, I argue the threshold is not a vibe; it is a deliberate line drawn on three measurable axes:

  • Impact. How much is at stake — in money, safety, legal exposure, or reputation? A $5 refund and a $50,000 contract are not the same decision.
  • Reversibility. Can the action be cleanly undone? Reversible actions can run autonomously and be corrected; irreversible ones — sending money, signing, deleting, publishing — deserve a gate.
  • Confidence. How sure is the system? A high-impact decision the model is unsure about is exactly where a human belongs; a routine one it has made correctly ten thousand times is not.

The rule writes itself once you score on these axes: high impact and low reversibility and/or low confidence → human approves first. Everything else → the agent acts and logs.

Why the line must be drawn before deployment

The cardinal error is deciding the threshold after something goes wrong. Set it in advance and it is a policy: defensible, consistent, and enforceable in code. Set it after an incident and it is a reaction: emotional, overbroad, and usually applied as "now a human checks everything," which kills the value of the agent and trains everyone to ignore the alerts. A written human review rule, agreed before launch, is what lets you expand autonomy responsibly instead of swinging between reckless and paralyzed.

It also has to be enforced by the system, not by good intentions. The threshold should be a hard control — the agent literally cannot commit an action above the line without a recorded human approval — and every approval (or auto-execution) should leave a tamper-evident trail. That is what turns "a human was supposed to check" into "here is who approved this, when, and on what evidence."

The takeaway for operators

Do not ask "should a human be in the loop?" — ask "where is the line, who set it, and is it enforced in code?" Score your AI's decisions on impact, reversibility, and confidence; put the gate where those axes turn dangerous; write the rule down before you deploy; and log every outcome. Done well, the human review rule does the opposite of slowing you down: it lets you safely automate the 95% of decisions that do not need a person, precisely because the 5% that do can never slip through.