The smartest thing a support bot can do is know when to stop.

Answering "how do I reset my password" is a solved problem. Point a language model at a help center, and it will handle the easy seventy percent of support tickets all day long. That part is not interesting anymore, and it is not where companies get hurt.

They get hurt on the other thirty percent. A customer asks for a refund, and a plain chatbot cheerfully explains the refund and sends it, no human anywhere. Someone types "delete my account and erase my data," and the bot actions the intent. A question lands that the docs simply do not cover, and the model, rather than admit it, invents a policy that sounds right. These are the expensive, trust destroying mistakes, and they are the reason most companies still keep humans on the front line instead of turning the AI fully on.

So I wanted to build the part everyone skips. Not another chatbot that answers, but the layer that decides whether the AI is even allowed to answer, and pulls a human in at exactly the right moment. I called it SupportIQ.

Knowing when not to answer is the product

The interesting question in support automation is not "can it answer," it is "does it know the limits of what it should decide alone." A system you can trust in front of customers has to do three things. Answer the safe, grounded questions instantly. Refuse to guess when the knowledge base cannot support a reply. And on anything sensitive, stop and let a person approve the message before it goes out.

That last behaviour, the human in the loop, is the one every demo waves at and almost none implement for real. A fake pause is easy. A pause that survives, that holds a half-finished conversation on a shelf until a human comes back minutes later and finishes it, is a genuine engineering problem. That is the part I wanted to get right.

SupportIQ, a state machine with a conscience

Under the hood SupportIQ is not a prompt with a few if-statements, it is a LangGraph state machine. Every customer message flows through the same graph: retrieve from the knowledge base, grade the situation, then branch to one of three outcomes.

Answer. Safe questions with a clear, grounded reply are answered on the spot, in plain language, with the sources kept on file. Human review. Anything sensitive, refunds, cancellations, security, data deletion, is drafted but not sent, it goes to an agent desk where a person approves, edits, or escalates it. Escalate. If retrieval is weak, or an answerability check finds the retrieved text does not actually cover the question, it hands off to a human instead of guessing.

The grade step is where the judgement lives. It reads three signals: how confident retrieval is, whether the retrieved passage genuinely covers the question, and whether the topic is on a sensitivity list that always requires a human. That middle signal mattered more than I expected. Keyword search will happily rank the pricing page highly for a question about next year's stock price, because they share the word price. An answerability gate, checking that the passage really addresses what was asked, is what stops a confident but irrelevant answer from ever being drafted.

The human in the loop is real, not a prop

When a ticket needs review, the graph does not fake a wait. It hits a real LangGraph interrupt, checkpoints the entire run, and stops. The draft, the sources, and the reason it was held all sit in the review queue. Later, a human on the agent desk clicks approve, edit, or escalate, and the very same conversation resumes from exactly where it paused, in a completely separate request. Proving that one flow, a paused run picked up and finished later, was the piece I was most careful about, because it is the difference between a demo and something you could actually put in front of customers.

Every answer the agent does send is grounded. It only states what a retrieved source supports, and a faithfulness check flags any reply that drifts beyond the evidence. The customer sees clean prose, the agent desk keeps the citations, which is exactly where each belongs.

Then I measured it against a plain bot

A claim like "safer" is worthless without a number. So I built a benchmark of 33 labelled questions across a 15-document knowledge base, and ran SupportIQ against the thing you get when you wire retrieval straight to a chat box, an answer-everything bot with no routing and no human. Both were scored on the same questions.

On the safe questions, the two tie, as they should. The whole gap is in the cases that cost money and trust. There were seven genuinely sensitive requests in the set, refunds, cancellations, a security report, an account deletion. The plain bot auto-sent an answer to every single one, with no human in the loop. SupportIQ held all seven.

On the questions the knowledge base could not support, the plain bot invented an answer every time, and SupportIQ escalated every time. Same model of the world, opposite instincts. The reliability does not come from a smarter language model, it comes from the graph around it: the routing, the grounding, and the permission to say "I need a person for this."

Three things I believe now

Refusing is a feature, not a failure. The most valuable thing an agent can learn is the edge of its own competence, so I scored refusal directly instead of hoping for it. A human in the loop has to be real. If the pause does not survive across requests, it is theatre, and LangGraph's checkpointed interrupts are what make it honest. And measure against the bot you would actually ship, not a strawman. The naive answer-everything baseline is what a rushed team deploys, and beating it on the cases that matter is the only comparison a stakeholder cares about.

It is open source

The full agent, the streaming console, the agent desk, and the reproducible benchmark are all in the repo. It runs offline with no key, or on a local model, or on OpenAI, with one environment variable.

If you work on agents, support automation, or applied AI governance, I would genuinely value your critique. I am always happy to talk shop, and always building toward something you can actually trust.