Agentic workflows that do not go rogue
Autonomy is not the goal. Reliable outcomes are. A look at the guardrails that make multi-step agents safe enough to put in front of a business process.
Amit Shivpratap Singh • • 7 min read
On this page
The demo-to-production gap
An agent that chains five tool calls is a compelling demo. The same agent running a thousand times a day against real systems is a different proposition, because the failure modes compound. If each step is 95% reliable, five steps land you at 77%.
That arithmetic is the whole problem. Production agents are not made trustworthy by better prompts alone; they are made trustworthy by shrinking the blast radius of each individual step.
Constrain the tools, not the model
The most effective guardrail is the narrowness of the tools you expose. An agent with a generic "run SQL" tool can do anything. An agent with "get open tickets for product X" can only do one thing, and you can validate its inputs.
Write tools the way you would write a public API: explicit parameters, validated types, predictable errors. The model is a caller, and callers should not be trusted with unbounded surface area.
# Narrow, validated, and impossible to misuse in interesting ways.
def get_open_tickets(product: str, limit: int = 20) -> list[dict]:
if product not in ALLOWED_PRODUCTS:
raise ValueError(f"unknown product: {product}")
limit = min(limit, 50)
return ticket_store.query(product=product, status="open", limit=limit)Put a human where the risk is
Human-in-the-loop is often treated as a failure to automate. It is better understood as placing review precisely where a mistake is expensive. Reading data needs no approval. Drafting a document needs none either. Sending that document to a supplier does.
A draft-then-approve pattern gets most of the time savings with a fraction of the risk, and it produces something valuable: a record of which suggestions humans accepted, which is the most honest evaluation set you will ever get.
Evaluate continuously, not once
Prompts drift, models get upgraded, source systems change shape. An agent validated in March is an unvalidated agent in September unless something is watching.
Keep a fixed set of representative cases with known-good outcomes and run them on every change. It does not need to be sophisticated — a few dozen cases and a pass rate you check before shipping will catch the regressions that matter.