A 50-session counterfactual replay after expanding ReclaimLLM's tool-output pipeline with native output shaping, provider canonicalization, deterministic edit receipts, and stateful deltas.
Executive result
3,278,328
of 13,420,338 text tool-result tokens removed
24.43%
text tool-output reduction (up from 16.46% in R1)
32.32%
equivalent tool-result capacity increase (up from 19.71%)
Replaying ReclaimLLM 0.1.35 over the organization's latest 50 eligible sessions reduced the text tool-result stream from 13,420,338 tokens to 10,142,010 tokens without re-running historical commands or calling a model. Across 18,510 non-image tool calls, 14,124 tool outputs (76.30%) were actively compacted, with 96.64% of result characters reachable by the transform core.
Every 1 million text tool-result tokens in this workload would require about 755,700 tokens after reduction. Within that portion of context, the remaining budget could carry roughly 323,200 additional tokens of equivalent original tool-output workload.
Capacity calculation
3,278,328 / 10,142,010 = 32.32%78.09% of all removed tokens (2,560,129 tokens) came from three new Round 2 mechanisms that compact native tool outputs directly.
exec, run_command, view_file, replace_file_content, grep_search), the tool's output is returned by the environment and appended to the conversation history as input context for the next LLM turn—and every subsequent turn until compaction. ReclaimLLM reduces those tool outputs at the hook boundary while retaining the exact raw result in session capture.Round 1 vs. Round 2 progression
In Round 1, savings were concentrated in standalone shell commands (rg, sed, and cd-prefixed commands). Round 2 repaired provider transcript capture for multi-block tool outputs and extended compaction to native file-view wrappers, echoed edit diffs, structured exec streams, and repeated stateful outputs.
| Measure | Round 1 (2026-08-09) | Round 2 (2026-10-26) | Change |
|---|---|---|---|
| Runtime implementation | 0.1.27 (3 mechanism families) | 0.1.35 (+4 new tool-output mechanisms) | Expanded native tool pipeline |
| Sessions considered / eligible | 97 / 61 (62.89%) | 70 / 50 (71.43%) | +8.54 pp eligibility rate |
| Reachable result-char coverage | Shell + basic reads | 96.64% | Broad native tool coverage |
| Original text tool-result tokens | 4,106,180 | 13,420,338 | 3.27× larger tool-output volume |
| Text tool-result tokens removed | 676,061 | 3,278,328 | 4.85× total tokens removed |
| Text tool-result reduction | 16.46% | 24.43% | +7.97 pp (+48.4% relative) |
| Equivalent capacity increase | 19.71% | 32.32% | +12.61 pp (+64.0% relative) |
Where the new savings come from
Four new or expanded Round 2 mechanisms—provider_output_shaping, provider_canonicalization, edit_receipt, and stateful_delta—accounted for 81.73% of all tokens removed (2,679,529 of 3,278,328 tokens).
Meanwhile, expanded H3_exec_compaction (including command-aware filters and read-only SQL row sampling) and range_cache contributed the remaining 598,799 tokens (18.27%).
| Mechanism | Round 2 status | Shaped calls | Original tokens (shaped) | Tokens removed | Share of savings | Reduction on shaped |
|---|---|---|---|---|---|---|
| provider_output_shaping | New in Round 2 | 1,079 | 1,828,102 | 1,060,575 | 32.35% | 58.01% |
| provider_canonicalization | New in Round 2 | 6,516 | 6,552,866 | 807,143 | 24.62% | 12.32% |
| edit_receipt | New in Round 2 | 1,634 | 859,463 | 692,411 | 21.12% | 80.56% |
| H3_exec_compaction | Expanded | 4,491 | 2,079,049 | 521,592 | 15.91% | 25.09% |
| stateful_delta | New in Round 2 | 229 | 157,185 | 119,400 | 3.64% | 75.96% |
| range_cache (H1) | Extended | 129 | 95,725 | 70,059 | 2.14% | 73.19% |
| lossless_search_path_folding | Shipped in R1 | 40 | 42,945 | 5,214 | 0.16% | 12.14% |
| legacy_compress / test_filter / hash_dedupe | Shipped in R1 | 6 | 4,180 | 1,934 | 0.06% | 46.27% |
| Total across shaped calls | All mechanisms | 14,124 | 11,619,515 | 3,278,328 | 100.00% | 28.21% |
New · 32.35% of savings
provider_output_shaping1,060,575 tokens removed
Shapes structured native tool outputs—specifically Codex exec streams prefixed with 'Script completed' / 'Chunk ID:' headers and Antigravity grep_search, find_by_name, and list_dir outputs—cutting shaped calls by 58.01%.
New · 24.62% of savings
provider_canonicalization807,143 tokens removed
Strips volatile capture timestamps ('Created At:' / 'Completed At:') and repetitive line-numbering / truncation wrapper notices injected into every view_file output across 6,516 file reads.
New · 21.12% of savings
edit_receipt692,411 tokens removed
When replace_file_content, multi_replace_file_content, or write_to_file succeeds, providers echo back the full [diff_block_start] patch the model just wrote. edit_receipt replaces the echoed diff with a compact SHA-256-verifiable receipt (80.56% reduction).
Expanded · 15.91% of savings
H3_exec_compaction521,592 tokens removed
Command-aware shell and test output compaction across 4,491 run_command and Bash calls (uv, pytest, grep, ls, python3, find, ssh), plus read-only supabase.execute_sql row sampling (>30 rows compacted to 12 head + 6 tail rows).
New · 3.64% of savings
stateful_delta119,400 tokens removed
Compares repeated shell, search, and listing outputs for the same semantic target against the preceding call in the session. When overlap is ≥60% and size ≥600 chars, it emits only [rclm state delta] with added/removed lines (75.96% reduction).
Extended · 2.14% of savings
range_cache (H1)70,059 tokens removed
Line-range read caching extended with build_antigravity_view_request to parse 'Showing lines X to Y' headers, replacing unchanged re-reads of previously viewed file ranges with a compact cache-hit reference (73.19% reduction).
Tool-output reduction by tool type
Across the 50 eligible sessions, 19,316 total tool calls were captured (18,510 non-image text tool calls and 806 image calls excluded from the text denominator). Four tool families—exec, view_file, replace_file_content, and run_command—generated 98.83% of all removed tokens.
| Tool name | Tool output role | Total calls | Original tokens | Tokens removed | Reduction | Mechanisms applied |
|---|---|---|---|---|---|---|
| exec | Codex command & script output | 1,805 | 1,776,515 | 1,031,313 | 58.05% | provider_output_shaping, hash_dedupe |
| view_file | File range inspection output | 6,725 | 6,655,416 | 877,202 | 13.18% | provider_canonicalization (807,143), range_cache (70,059) |
| replace_file_content | Echoed file patch diffs | 1,656 | 860,516 | 692,411 | 80.46% | edit_receipt |
| run_command | Shell & test runner output | 6,412 | 2,778,137 | 639,000 | 23.00% | H3_exec_compaction (521,592), stateful_delta, path folding |
| grep_search | Repository pattern search output | 555 | 415,368 | 26,583 | 6.40% | provider_output_shaping, stateful_delta |
| find_by_name | File path discovery output | 189 | 49,810 | 7,933 | 15.93% | provider_output_shaping, stateful_delta |
| list_dir | Directory listing output | 63 | 16,860 | 3,886 | 23.05% | provider_output_shaping, stateful_delta |
| Other text tools | MCP calls, metadata, uncovered | 1,105 | 867,716 | 0 | 0.00% | Passed through unchanged |
| All text tool calls | Non-image tool outputs (14,124 shaped) | 18,510 | 13,420,338 | 3,278,328 | 24.43% | 76.30% of text tool calls actively compacted |
run_commandWithin the 639,000 tokens removed from run_command tool outputs, package/test runners (uv, pytest) and search/inspection commands (grep, ls, python3, find) dominated:
| Command family | Tokens removed | Typical tool-output noise compacted |
|---|---|---|
| uv | 171,413 | Package resolution, virtual-env sync logs, and test runner wrappers |
| pytest | 120,724 | Passing test dots, fixture setup logs, and repeated assertion tracebacks |
| grep | 76,262 | Unscoped multi-file search output and repeated path prefixes |
| ls | 60,119 | Long directory listings across repeated turns |
| python3 | 54,987 | One-off diagnostic scripts and data inspection dumps |
| find | 37,681 | Deep directory tree enumerations |
| ssh | 34,617 | Remote deployment and container inspection output |
| cd | 19,217 | cd <dir> && ... compound shell commands |
| git | 17,373 | Status, diff, and log inspection commands |
| rg | 9,306 | Ripgrep search results |
Corpus eligibility & provider breakdown
In Round 1, Gemini/Antigravity tool outputs saw only a 2.29% reduction because wrapped view_file, replace_file_content, and run_command envelopes were not yet parsed by native transforms. In Round 2, Gemini/Antigravity reduction rose to 19.87% (2,148,670 tokens removed).
Codex reduction rose from 40.10% to 45.87% (84.74% equivalent capacity increase) after repairing custom_tool_call_output array ingestion, while Claude rose from 9.86% to 27.24%.
| Eligibility stage | Sessions |
|---|---|
| Considered | 70 |
| Eligible and replayed | 50 |
| Excluded | 20 |
Of 20 exclusions, 9 lacked ≥5 replay-eligible calls, 9 had incomplete session_state, 1 lacked enough text-result tokens, and 1 exceeded the allowable unresolvable share. Eligible share: 71.43%. Sample confidence: full.
| Source | Original tokens | Tokens removed | Reduction | Capacity increase |
|---|---|---|---|---|
| Gemini / Antigravity | 10,813,804 | 2,148,670 | 19.87% | 24.80% |
| Codex | 2,248,399 | 1,031,313 | 45.87% | 84.74% |
| Claude | 267,484 | 72,863 | 27.24% | 37.44% |
| Unattributed | 90,651 | 25,482 | 28.11% | 39.10% |
| Total | 13,420,338 | 3,278,328 | 24.43% | 32.32% |
Unresolvable calls dropped to 75 across the entire 50-session corpus (0.42% of eligible calls), with 96.64% of result characters reachable by the transform core.
replay_compare)| Pipeline stage configuration | Tokens removed | Reduction | Reachable char coverage |
|---|---|---|---|
| shell_compaction stage alone (native tool transforms + edit_receipt + canonicalization + H3_exec) | 3,175,210 | 23.66% | 93.85% |
| stateful_delta stage alone | 119,400 | 0.89% | 1.17% |
| range_cache (H1) stage alone | 70,059 | 0.52% | 48.23% |
| hash_dedupe stage alone | 1,338 | 0.01% | 96.66% |
| Combined 4-stage pipeline (range_cache → stateful_delta → shell_compaction → hash_dedupe) | 3,278,328 | 24.43% | 96.64% |
Because range_cache and stateful_delta execute before shell_compaction in the combined pipeline, some calls claimed early by caching or delta differencing would otherwise have been partially compacted by shell_compaction; the combined row reports the exact non-overlapping net reduction.
Window sensitivity check
To verify that the Round 2 result is not an artifact of scanning a 60-day window to reach 50 eligible sessions, Replay was also executed across 45-day and 30-day rolling windows.
| Rolling window | Considered | Eligible | Original tokens | Tokens removed | Reduction | Capacity increase |
|---|---|---|---|---|---|---|
| 60 days (target 50 eligible) | 70 | 50 | 13,420,338 | 3,278,328 | 24.43% | 32.32% |
| 45 days | 73 | 48 | 11,835,147 | 3,135,811 | 26.50% | 36.05% |
| 30 days | 62 | 42 | 9,789,727 | 2,032,805 | 20.76% | 26.21% |
Interpretation
Supported
Across 50 sessions (~13.4M tool-output tokens), ReclaimLLM removed ~3.2M tokens from tool outputs (~24% reduction)—letting the same token budget carry ~32% more coding workload.
Not supported
Replay never invokes a model. It cannot observe path changes, retries, or turn-count effects, which may make real-world impact lower—or negative. Prompt caching can also change billable tokens without changing the raw tool-output reduction measured here.
Explore the evidence
Compare this Round 2 analysis with the original Round 1 baseline, explore each context-compression mechanism in detail, or install rclm to run read-only Replay against your own organization's captured sessions.