The Economics of an Agent Loop

The demo cost nothing to run and the production version cost a fortune, and for a while I couldn’t see why. Same agent. Same model. The difference was the loop. In the demo it took four steps. In production, against messy real data, it took nineteen, and each of those steps carried the full and growing transcript with it. Cost in an agent loop is not a line item you check at the end. It is a property of the architecture, and it compounds per step in a way that is easy to miss until the bill arrives.

I now design agents around three budgets from the first sketch: tokens, time, and error probability. They are linked, they all scale with the number of steps, and treating them as first-class constraints changes what I build.

Cost is quadratic-ish, not linear

Here is the trap. People reason about agent cost as “steps times cost-per-step” and assume it’s linear. It isn’t, because in most loops each step carries the accumulated context of every prior step. Step one is cheap. Step fifteen pays to re-process fourteen steps of history. The transcript grows, and you pay for the whole transcript on every call.

Step	New tokens	Context carried	Tokens billed this step
1	800	800	800
5	800	~5,000	~5,000
10	800	~11,000	~11,000
19	800	~22,000	~22,000

The new work per step is flat. The bill per step climbs because the context climbs. A loop that runs twice as long doesn’t cost twice as much. It costs more, because the later steps are each individually fatter. This is exactly why context discipline — distilling tool outputs, compressing history — is also a cost lever, not just a reliability one. Every token you keep out of the carried context is a token you stop paying for on every subsequent step.

The cheapest step is the one the agent doesn’t take. The second cheapest is the one that doesn’t drag the whole transcript along.

Time is the budget users actually feel

Tokens hit your margin. Latency hits the user, and they are not patient. Each agent step is a full model round-trip, often hundreds of milliseconds to a few seconds depending on how much it has to read and generate. A nineteen-step loop, even at a fast 600ms per step, is over eleven seconds of staring at a spinner. On HANNA, my voice agent, that’s not a slow UX, it’s a dead call. A human will not hold the line through eleven seconds of silence.

So latency reshapes the design directly:

• Step count is a hard budget, not a nice-to-have. Fewer, fatter steps usually beat many thin ones for wall-clock time, even if the token count is similar.
• Parallel independent work. If three tool calls don’t depend on each other, fire them together. Three sequential 600ms calls is 1.8 seconds; three parallel is 600ms.
• Stream what you can. The user perceiving progress is worth more than the loop technically finishing sooner.

Error probability is a budget too

This is the one people don’t put on a spreadsheet, and it’s the most important. Every step is a chance for the agent to go wrong — pick the wrong tool, misread a result, drift off the goal. If each step is, say, 97% reliable, that sounds great until you compound it. Twenty steps at 97% each is roughly 0.97²⁰, about a 54% chance the whole trajectory is clean. Cut the loop to eight steps and you’re at about 78%. The math is unforgiving: every step you add is a tax on the probability the agent finishes correctly.

P(clean trajectory) = per_step_reliability ^ steps
0.97 ^ 8  ≈ 0.78
0.97 ^ 20 ≈ 0.54

This is why shorter loops aren’t just cheaper and faster. They’re more correct. A planning loop that resolves a task in six checkpointed steps will beat a reactive loop that wanders through twenty, on all three budgets at once. The architecture decision and the economics decision are the same decision.

The moves that actually move the budgets

When an agent is too expensive, too slow, or too error-prone — and it’s usually all three together — these are where I go, roughly in order of payoff:

Cut steps. The biggest lever by far. Can a coarser tool collapse three calls into one? Can a plan replace exploratory wandering? Fewer steps helps every budget simultaneously. I covered tool granularity elsewhere; this is where that decision pays off in dollars.

Trim carried context. Distill tool outputs before they enter the transcript. Summarize and compress long histories. Every token you stop carrying is a token you stop paying for on every later step.

Right-size the model per step. Not every step needs the frontier model. Routing, classification, simple extraction — a smaller, cheaper, faster model handles these fine, and I save the expensive model for the steps that genuinely reason. A mixed-model loop can cut cost meaningfully without touching quality where it matters.

Cap and gate. A hard step limit so a stuck agent can’t run up an unbounded bill. And a kill on the loop if it starts repeating, because a looping agent is pure waste on all three budgets at once.

Design to the budget, not away from it

The reframe that stuck with me: an agent is not a magic box you optimize after the fact. It’s a loop whose cost, latency, and error rate are baked into its shape the moment you choose the control flow and the tools. A ten-step task and a four-step task are different products, not the same product tuned differently.

So I put the budgets on the table at design time. Before I build, I ask: how many steps should this realistically take, how long can the user wait, and how many compounding chances to fail can I afford. Those three numbers constrain the architecture as hard as any requirement. Build the simplest loop that fits inside them. Autonomy is a cost, and the loop is where you pay it — token by token, second by second, and one compounding chance of being wrong at a time.