The word “agent” has been stretched until it means almost nothing. A chatbot with a function call is an agent. A cron job that summarizes email is an agent. A 2,000-line prompt is an agent. When a term covers everything, it stops being useful for deciding how to build.
I build these systems for a living — voice agents that cold-call, retrieval pipelines that reason over federal award data, an ERP that operates itself through tool calls — and I’ve had to settle on a working definition that actually changes my architecture. Here it is.
An agent is software where the model decides the control flow, not just the content.
That is the whole distinction. Everything else — tools, memory, planning, multi-step loops — is downstream of it.
Control flow is the dividing line
In ordinary software, a human wrote the branches. if payment.failed → retry. The program can only do what its author anticipated. A language model embedded in that program is a component: it fills a slot the author already carved out. You ask it to classify, extract, or rewrite, and you take the output and feed it into a path you designed.
In an agentic system, the model sits one level up. It looks at the current state, decides which tool to call, reads the result, and decides what to do next — and the “next” was not enumerated in advance by a human. The loop is fixed; the trajectory through it is not.
This is why the same model call can be agentic or not depending entirely on where you put it. The model didn’t change. The authority did.
The minimal agent loop
Strip away the frameworks and every agent is the same loop:
state = initial_context
while not done:
decision = model(state, available_tools)
if decision.is_final:
return decision.answer
result = execute(decision.tool, decision.args)
state = state + decision + resultFour moving parts: a model that emits a decision, a set of tools it can choose from, an executor that runs the chosen tool, and a state object that accumulates everything seen so far. That’s it. A graph framework, an agent SDK, a hand-rolled while loop — all of them are this shape with different ergonomics.
Once you see the loop, the engineering questions get sharp:
- What’s in
state? Too little and the model repeats itself; too much and it drowns in its own history and the token bill explodes. - When is
donetrue? A model that can’t reliably decide it’s finished will loop forever or quit early. - What’s in
available_tools? Every tool you add widens the decision space and raises the odds of a wrong turn.
These are not prompt problems. They are systems problems wearing a prompt’s clothing.
The autonomy spectrum
“Agent vs. not an agent” is a false binary. Real systems sit on a spectrum of how much control flow they hand to the model:
| Level | Who decides next step | Example |
|---|---|---|
| 0 — Fixed | Human author | Classic ETL with an LLM enrichment step |
| 1 — Routed | Model picks one of N predefined branches | Intent classifier → fixed handler |
| 2 — Tool-calling | Model chooses tools in a single turn | RAG with model-selected retrieval |
| 3 — Looped | Model iterates tools until it decides done | Research agent, coding agent |
| 4 — Planning | Model decomposes goals into subgoals it then executes | Multi-step autonomous workflows |
| 5 — Initiative | Model decides when to act at all, unprompted | Durable agents reacting to events |
Most production “agents” today live at level 2 or 3. The marketing implies level 5. The gap between them is where projects die — teams design for the demo (level 3 looks magical) and then discover they needed level 2’s predictability to pass a compliance review.
I almost always build for the lowest level that solves the problem. Autonomy is a cost, not a feature. Every step up the ladder buys flexibility and pays in unpredictability, latency, token spend, and debugging difficulty.
Why this definition earns its keep
A definition is only worth having if it makes decisions for you. This one does:
It tells you when you don’t need an agent. If you can enumerate the branches, enumerate them. A routed system at level 1 is faster, cheaper, fully testable, and won’t surprise you in production. Most “let’s build an agent” requests are actually level-1 problems in disguise. Reaching for an agent loop there is how you turn a 50ms deterministic path into a 4-second, $0.08, occasionally-wrong one.
It tells you what to instrument. Because the model chooses the path, you cannot reason about behavior from the code alone — the code only describes the space of behavior. You have to capture actual trajectories: every decision, tool call, and result, on every run. An agent without trajectory logging is unobservable by construction. This is non-negotiable; I treat it the way a backend engineer treats request logs.
It tells you where reliability comes from. In fixed software, reliability is correctness of the branches. In agentic software, reliability is the quality of the model’s decisions across many trajectories — which means your reliability story is an evaluation story, not a unit-test story. You measure distributions of outcomes, not single assertions.
It tells you the real risk surface. The danger in an agent isn’t a bad string. It’s a bad action — a tool call that sends an email, moves money, deletes a record. The moment the model controls flow, the blast radius is the union of everything your tools can do. That reframes security from “validate the output” to “constrain the action space and gate the irreversible ones.”
The trap of premature agency
The most expensive mistake I see — and made myself early — is reaching for level 4 or 5 because it’s intellectually exciting, when the problem was a level-2 problem. You end up debugging emergent behavior in a system that didn’t need to be emergent. The model takes a creative route to a goal you could have hard-coded, and now you’re writing guardrails to prevent the creativity you introduced on purpose.
The discipline is to keep asking: does the model actually need to decide this, or do I just not want to write the branches? If it’s the latter, write the branches. Your future self, staring at a non-reproducible failure at 2 a.m., will thank you.
Where I land
An agent is software that delegates control flow to a model. That delegation is powerful and occasionally necessary — some problems genuinely have a decision space too large or too dynamic to enumerate, and for those, nothing else works. But it is never free, and it is rarely the first thing you should reach for.
Build the simplest thing on the autonomy spectrum that solves the problem. Instrument the trajectory, because the code no longer tells you what the system does. Gate the irreversible actions, because the model now chooses them. And stay honest about which level you actually need — the marketing rewards level 5, but production rewards the lowest level that ships. That honesty is the design principle underneath Vontra, the AI-native business operating system I’m building: agents that operate the business run at the lowest autonomy level the task allows, and every irreversible move is gated.
Everything else in this archive — agentic RAG, autonomous operations, self-running ERP — is a specific answer to one question: given that the model now controls the flow, how do we make that safe, observable, and good?