Most of the agent bugs I have chased were not reasoning bugs. They looked like reasoning bugs. The model would say something dumb, contradict an earlier step, or forget a constraint I had stated in plain language. The reflex is to blame the model and reach for a bigger one. Nine times out of ten the real problem was that the relevant fact was not in the window when the model needed it, or the window was so cluttered the model couldn’t find it. The reasoning was a costume. The failure was context.
That reframing changed how I build. An agent is a function from context to next-action. If the context is wrong, the action is wrong, and no amount of prompt tuning fixes a state-management problem.
The window is a workbench, not a warehouse
The context window is not storage. It is the model’s working surface for one step. Everything on it competes for attention, and attention degrades as the surface fills. Bury the one constraint that matters under twelve turns of tool output and the model will sail right past it, even though it is technically “in context.”
This is the failure people miss because the token count says everything is there. Presence is not salience. I have had agents ignore a hard rule sitting 8,000 tokens up the transcript, then obey it perfectly once I moved it to the bottom of the prompt where it was fresh. Same model, same words, different position. The window has a geography and the model reads it unevenly.
An agent does not fail because a fact is missing from context. It fails because the fact is present but not salient at the step where it matters.
What earns a place in the window
My default is exclusion. A piece of information has to justify its slot. The questions I ask:
- Does the model need this to decide the next step? Not the task overall, the next step. If not, it can live outside the window and get pulled in on demand.
- Is it raw or distilled? A 4,000-token API response usually carries 40 tokens of signal. Distill it before it lands in context.
- Will it still be true in three steps? Stale state is worse than no state, because the model trusts it.
In a federal-contractor intelligence platform I built, when an agent pulls a contractor’s award history from public federal award data, the raw payload is enormous. Dropping it whole into context poisons every later step. So the tool returns a summary — total obligated, top agencies, most recent award — and stashes the full record behind an id the agent can re-fetch if it genuinely needs detail. The window holds the conclusion, not the evidence.
The four ways context goes wrong
After enough incidents I sort context failures into four buckets, and the fix differs for each:
| Failure | What it looks like | Fix |
|---|---|---|
| Poisoning | one bad fact early corrupts every later step | validate tool outputs before they enter context |
| Distraction | so much noise the signal gets lost | distill, summarize, drop raw payloads |
| Confusion | irrelevant-but-plausible info pulls the model off-task | scope retrieval tighter to the current step |
| Staleness | model acts on state that changed | timestamp state, re-fetch before irreversible acts |
Poisoning is the nastiest because it compounds. One hallucinated entity id from step one becomes the false premise of steps two through ten, and the model reasons flawlessly from a wrong fact. I now treat tool outputs as untrusted input the way I’d treat user input to a web form. Validate before it enters the working set.
Managing the trajectory, not just the prompt
For short agents you can let the full transcript ride. For anything long that breaks, because the transcript grows without bound and the early steps fade. The discipline is to actively manage what the agent carries forward between steps, not just what you put in the system prompt.
Three moves I rely on:
Summarize and compress. Past a threshold, I replace the raw step history with a running summary: what we’ve established, what’s still open, what constraints are live. The summary is regenerated, not appended, so it stays tight. This is the single biggest lever on long-trajectory reliability.
Externalize state. The agent does not have to hold everything in its head. A scratchpad, a structured working memory, a small JSON object I thread through each step — these let the agent write down a decision once and reference it without re-deriving it. It mirrors how a planning loop checkpoints its steps; the plan itself is externalized state the reactive inner work reads from.
Pull, don’t push, the big stuff. Reference documents, full records, long histories live outside the window behind a tool. The agent retrieves the slice it needs for the current step instead of carrying the whole thing through every step.
// Bad: the whole record rides along forever
context += full_client_record // 6k tokens, relevant to step 2 only
// Good: distilled now, full detail on demand
context += summarize(client_record) // 80 tokens
// agent calls get_client_detail(id) only if it actually needs moreWhy this is upstream of everything
Context engineering is not a tuning step you do at the end. It is upstream of tool design, because what a tool returns becomes context. It is upstream of your control loop, because a reactive loop accumulates context fast and a planning loop has to decide what each step inherits. It is upstream of cost, because the window you fill is the window you pay for on every call.
The agents I trust in production are not the ones with the cleverest prompts. They are the ones where I can point to any step and explain exactly why the context at that step is correct, current, and tight. That discipline is the spine of Vontra, the AI-native business operating system I’m building — every agent action runs off context I can account for, not a transcript I’m hoping the model parses correctly. The model does the reasoning. My job is to make sure that when it reasons, it is looking at the right things, and only the right things. Get that wrong and you will spend weeks debugging “reasoning” that was never the problem.