Writing up that energy-use measurement left me with a number I couldn’t stop thinking about: 96% of the 3.2 billion tokens processed weren’t new work, they were the agent re-reading its own accumulated context at every one of 14,000 steps. That only makes sense if you actually understand what “context” is doing, so here’s the plain version.
Context is everything the model can see for a given call — nothing more, nothing less. Not memory in any lasting sense; more like short-term working memory that gets rebuilt from scratch every time. It includes the system prompt, the conversation so far, any files the agent has read, the output of any tools it’s called, and whatever instructions you gave it. The model has no access to anything outside that window — it can’t recall yesterday’s session unless something explicitly fed that history back in, which is exactly the problem claude-mem exists to paper over.
Here’s the part that explains the 96% figure. A model doesn’t retain a running memory between calls the way a person does — every single API call is stateless, and has to include the full context again for the model to have any idea what’s going on. So when a coding agent reads a file, calls a tool, checks the result, and decides on a next step, that entire history — the original task, every file read so far, every tool result — gets sent again on the next call, and the one after that, and the one after that. A session with 100 steps doesn’t send 100 independent small requests; it re-sends a context that’s grown a little bigger each time, 100 times over. That’s structurally why an agentic session burns so many more tokens than a single chat exchange: it’s not doing 100x the thinking, it’s re-reading its own accumulated notes 100 times to stay oriented.
Why nobody’s just fixed this: some of it does get optimized — prompt caching specifically exists to make repeated context cheaper to reprocess rather than free, which is part of why cache-read tokens are usually billed at a fraction of fresh input tokens. But caching makes re-reading context cheaper, it doesn’t eliminate the re-reading itself, because the fundamental constraint — a stateless model call needs the full picture every time — doesn’t go away. MCP helps by making it cheaper to fetch only the specific context needed rather than dumping unrelated tools into every call, but it’s addressing what gets included, not the resend-everything-every-step mechanic underneath.
The practical upshot, if you’re the one paying per token: the length of a session matters more than the number of prompts you type into it. A single long-running agentic task that takes 40 steps to finish is going to cost meaningfully more than the same total work split into a few short, focused sessions — not because the model does more thinking, but because a longer session means more context gets carried forward, and resent, at every step along the way.
