Human-in-the-Loop: When Should AI Hand Off to a Human?

An AI setter should hand off to a human when the person asks for one, is upset, raises something the AI isn't allowed or able to handle (medical, legal, complaints, unusual requests), or is a high-value lead you want a person to speak to. "Human in the loop" also means something broader: letting people approve what the AI sends until you trust it, and reviewing its work over time.

Getting this right is what makes AI safe to use with real customers. I put a lot of effort into it in ReplySetter, because the alternative, an AI that never lets go, is how businesses get embarrassed.

The two kinds of human in the loop

1. Approvals: the AI proposes, a human decides. Good for launching and for sensitive actions.
2. Handoffs: the AI hands the whole conversation to a person when it should.

Approvals and autonomy levels

Most AI setters offer some version of these autonomy levels. In ReplySetter they're called:

• Suggest: the AI proposes every reply and action; a person approves, edits or rejects each one.
• Draft: the AI writes drafts in your inbox; a person sends them.
• Auto: the AI sends on its own, within the rules you set.

The way I recommend using them:

1. Launch in suggest. Approve every reply for the first few days.
2. Track how many you approve unchanged. When it's nearly all of them, you're ready.
3. Move to auto for routine conversations.
4. Keep sensitive actions on approval if you want, such as refunds or cancellations.

All pending actions wait in one approvals queue across every inbox. See human in the loop.

When the AI should hand off

The person asks for a human

Always honour it. "Can I talk to a real person?" should never get "I can help you with that!"

The person is upset or angry

Complaints, frustration, threats to leave a bad review. A human apology lands better.

Sensitive topics

• Medical symptoms or emergencies.
• Legal questions.
• Financial hardship.
• Bereavement or personal crises.

Things the AI can't or shouldn't do

• Custom pricing or discounts.
• Exceptions to policy.
• Partnership, press or job inquiries.
• Anything not covered by the knowledge base.

High-value leads

Big accounts, VIPs, or leads above a budget threshold you'd rather a senior person handled.

The AI is going in circles

Several back-and-forths without progress usually means the AI doesn't understand. Hand off.

How to set handoff rules

In ReplySetter, handoffs are usually set up as scenarios that apply anywhere in the flow:

When: the person asks for a human, is upset, or asks about something not in the knowledge base.

Do: reply that someone from the team will follow up shortly, then hand off.

A handoff:

• pauses the AI on that conversation,
• assigns it to a person (or a team inbox),
• adds a note explaining why,
• notifies your team.

There's also a Stop step in flows that ends the conversation and optionally hands it to a teammate.

Make the switch smooth

What the lead should experience:

• A clear, kind message: "I'll get Sarah from our team to help with this, she'll reply here shortly."
• The same thread, not a new channel.
• No repeating themselves: the human should see the whole conversation and a short summary.

What your team needs:

• A notification they'll actually see.
• Context: why it was handed off and what the lead wants.
• A way to give the conversation back to the AI later if it's routine again.

When humans reply, the AI should step back

If a team member replies in a conversation, the AI should notice and stay out of it, so they don't talk over each other. In ReplySetter, you can pause or resume the AI per conversation at any time.

Review: the third part of human in the loop

Even on full auto, keep a human reviewing:

• Weekly: read a random sample of conversations.
• Look at handoffs: were they necessary? Could the AI have handled some with better knowledge?
• Look at non-handoffs: were there conversations where it should have handed off?
• Improve: add knowledge, tighten instructions, adjust rules.