← Back to Blog

deep-dive

By ReclaimLLM Team

How we cut coding-agent tokens without breaking the prompt cache

How we reduce Claude Code and Codex token usage with append-only context compression that keeps the prompt cache intact, and what we measured wrongly.

How we cut coding-agent tokens without breaking the prompt cache

Executive summary

  • Most of a coding agent's tokens are tool results: file reads, command output, search results. In our research corpus they were about 56% of captured tokens, and native file reads alone were about half of those.
  • Every ReclaimLLM savings method is a deterministic rule in the agent's own hooks. There is no LLM in the hot path, and anything a rule doesn't recognize passes through unchanged.
  • We shape each result once, as it enters context, and never rewrite history. With prompt caching, rewriting earlier turns usually costs more than it saves.
  • We built the methods in rounds, each driven by data: first obvious noise, then repeats, then a replay harness to measure ourselves, then a corpus study that changed what we build next.
  • The biggest lesson was about measurement. A "tokens saved" counter is not savings until you check what the model actually received. We now report saved and potential separately.

Where the tokens go

A coding agent re-sends its whole conversation on every turn. A file it read on turn 3 is still in context on turn 300. So the cost of a tool result isn't its size once; it's its size times every later turn it rides along on, billed mostly at the cached-read rate.

When we broke down 144 captured sessions by content type, tool results were 55.8% of tokens (59.2% once you weight by how long each token stays in context). Native file reads alone were 49.1% of result tokens. Tool inputs, mostly edit and write payloads, were next at about 28%. User prompts were 1.5%.

That breakdown decides where savings can come from. Prompt tweaks and shorter answers are a rounding error next to what agents read and run.

Teams feel this too. On the Cursor Forum, an enterprise user asked for per-session and per-team AI cost budgets, and Cursor staff confirmed that hooks get no cost data today.

Rule one: shape once, on the way in

Providers cache the conversation prefix. On the Claude sessions where we had per-request usage, 98.5% of input tokens were cache reads. Cache reads are cheap (about 0.1x the base input price), cache writes are not (1.25x on the 5-minute cache, per Anthropic's prompt caching docs), and any change to an earlier turn invalidates everything after it.

That gives one design rule we haven't broken: append-only. Every method changes a tool result before the model first sees it. Nothing is evicted or rewritten later. In our cost model, evicting old results every turn raised input cost by about 85%, even though it removed lots of tokens. Capping new results as they arrive reduced cost.

The other rules followed:

  • Deterministic. No summarizer model on the hot path. The same input gives the same output, so it can be tested and replayed.
  • Fail open. A parser error, an unfamiliar command, a failed command, or an ambiguous shape returns the original result.
  • Recoverable. When a result is shortened in a lossy way, the exact original is stored locally, and the agent gets a handle to read it back.

The methods, in the order we built them

Round 1: obvious noise (spring 2026)

The first mechanisms targeted output that is long and mostly repetitive:

  • Native read caps. Unbounded reads of long files are capped at 200 lines.
  • Search shaping. Long grep and rg results become match counts, per-file counts, and representative lines. The agent can always run a narrower search.
  • Exec compaction. Git, directory listings, and generic long output keep a bounded head and tail plus an omitted-line count.
  • Loop breaker. Repeated identical calls get corrective context, then a permission prompt. It never blocks silently.

These came from looking at sessions by hand. You don't need a study to see that a 4,000-line git log is mostly not read.

Round 2: stop paying twice (summer 2026)

Next we looked for content the agent already had:

  • Range cache. We track which line ranges of each file the model has already received. A full re-read becomes a one-line "unchanged since turn N" notice, and a partial overlap returns only the new lines. File hashes and edits invalidate the cache. Serving the wrong lines is worse than missing a cache hit, so ambiguous commands pass through.
  • Hash dedupe. A large result identical to an earlier one becomes a reference to the first.
  • Test output filter. Pytest, Jest, Vitest, Go, and Cargo runs keep failures, tracebacks, and the summary, and drop pages of passing tests.

Round 3: measure ourselves (August 2026)

By then we had a dozen mechanisms and dashboard counters that looked good. We didn't trust them, so we built Replay: read-only tools that run the exact shipped transforms over captured sessions with a real tokenizer, and report how many tokens each method would remove on your sessions. It reports tokens only, never dollars, and says what it can't tell you.

Replay changed what we built next, because it showed where savings were not happening. For a step-by-step guide to measuring this on your own sessions, see measuring token waste in Claude Code and Codex.

Round 4: provider-aware shaping (September 2026)

Replay showed large amounts of wrapper text and echoed content:

  • Provider canonicalization. Strip known boilerplate (timestamps, execution headers, repeated path prefixes) from Codex and Antigravity result wrappers.
  • Stateful deltas. A repeated command against the same target returns only what changed since the last run.
  • Edit receipts. After a successful edit, the tool often echoes back the full diff the agent just wrote. We replace it with a receipt: target, hunk count, line counts, and a digest.
  • Lossless path folding. Fold repeated path prefixes in search results, and only keep the fold after expanding it and checking it reproduces the original exactly.

On a 50-session corpus these took tool-output reduction from about 16% to about 24%, with Codex around 46%. The full numbers are in the round 2 token savings case study.

Round 5: a corpus study (November 2026)

This round started from the data rather than from ideas. We replayed the shipped engine over 159 captured sessions from our own team, sized new techniques against the remainder, and priced everything with prompt caching. The findings:

  1. The rules would remove 19.5% of text tool-result tokens, but only 0.5% of calls were actually compressed at capture. Most sessions ran with mechanisms in observe mode, or without hooks enforcing.
  2. Claude's biggest token class was barely touched. Native Read is about 60% of Claude's result tokens, and the read-cache replacement wasn't reaching the model. More on that below.
  3. Savings depended heavily on the client. Codex results shrank 35% in replay, Claude 8%.
  4. Some popular ideas don't pay once caching is priced. Eviction and aggressive compaction remove a lot of tokens and save roughly zero dollars, or cost more.

That gave a ranked build list. We shipped the low-risk, append-only items first:

  • Cross-session outline. When a session re-reads a large file from the top and another session already read the same unchanged content (the hash is re-checked on disk), the model gets a symbol outline with real line numbers. Then it reads only the range it needs. In one end-to-end run with the real Claude CLI, a 677-line file cost about 13k tokens to read in full and 689 the second time (outline plus one targeted read), with a correct answer. If the model re-reads the file in full anyway, we record a negative saving. It ships in observe mode by default.
  • Repeated log-line collapse. Retry loops and pollers print the same line hundreds of times with only numbers changing. We keep the first line and note how many similar lines followed. Tables, CSV, JSON, and key/value lines are never folded; an early dogfooding run caught us folding a numeric results table, and the rule was tightened.
  • Blob placeholders. Long base64 and hex runs become a placeholder with length and digest. Commit hashes and SHA-256 digests are kept, because agents quote them.

What we got wrong

The corpus study found a defect that reached past savings. Claude Code applies a replacement tool result only when it has the same shape as the original: file content for Read, stdout and stderr for shell commands. We were sending plain strings, which Claude Code silently ignored. As a result:

  • Read-cache replacements were logged as applied, but the model still got the full file. Our savings figure was too high.
  • Redacted DLP output for Read and cat was dropped the same way, so unredacted text reached the model in those cases.

Both are fixed. Replacements keep the original shape, and we verified with the real Claude CLI that the model receives the redacted and cached text. We also fixed two smaller correctness gaps: cache state now resets when the conversation is compacted, and each subagent keeps its own state, so an "unchanged since turn N" notice never points at content the model no longer has.

Saved versus potential

The fix changed how we report numbers. Every savings event now records whether it was applied (the model got the shorter result) or only measured in observe mode. The enterprise tokens page and the weekly digest show two figures:

  • Saved: tokens the model never received.
  • Potential: what observe-mode mechanisms measured but didn't apply.

For older data recorded before that flag existed, we count savings as applied only when we can be sure. Otherwise the saved figure is 0. Some customers' reported savings dropped as a result.

What we still don't know

  • Tokens aren't dollars. Our runtime counts are character-based estimates, and removing a token saves different amounts depending on when it entered the cache. Our models put shipped rules in the low single digits of percent of billed input for Claude-like sessions. We haven't measured this against real bills yet.
  • Fewer tokens can mean more turns. If an outline makes the agent come back for the file, the token cost shows up as a negative saving, but the extra round trip doesn't. Task success, turn count, and retries are the real test, and we haven't run a controlled study of them.
  • The research corpus is our own team. 159 sessions from one team is enough to rank ideas, not to promise a percentage on your workload. That's why Replay exists: run it on your sessions before you enable anything.
  • Some things are still on the list. A result cap with a recall pointer for tools that have no rule today, and a lower context-handoff threshold, both sized well in our models. We haven't shipped them, because their quality risk isn't measured yet.

How to try it

Install the hooks (see cost savings), leave the newer mechanisms in observe mode, and compare the saved and potential totals on your own sessions. Then use Replay to see which methods would matter for your workload before you turn on enforcement. The full mechanism reference, including which clients enforce which methods, is in the context compression docs.

If a method hid something your agent needed, or your numbers look wrong, send us the session. Most of the rules described here were tightened that way.