Context Compression for Coding Agents: Keep Signal, Cut Tokens
•
7 min read

Context Compression for Coding Agents: Keep Signal, Cut Tokens

Coding agents resend full history on every call, long context hurts answers, and compaction loses detail. What survives, and the cheapest ways to cut tokens.

Context compression for coding agents means keeping the smallest set of tokens that still carries enough signal for a correct answer. Long sessions fill the window with file reads and tool output, every request resends it all, and fuller context often performs worse, so cutting tokens helps accuracy and cost.

A long coding session feels productive until the agent starts missing details it handled fine an hour ago. The cause is usually not the model.

The fix starts with three facts: what lives inside that context, why fuller context often performs worse, and which token cuts the official Claude Code docs endorse before any third-party tool.

What fills the context window

The Claude Code docs on the context window describe the starting context before you type anything: the system prompt (around 4,200 tokens in the docs example), project memory such as CLAUDE.md, environment details, MCP tool names, skill descriptions, and global rules. Every file read, grep result, and test run then adds more, and the docs state plainly that file reads account for most of the growth.

The billing trap is structural. The agent keeps no memory between API calls, so each prompt resends the full conversation plus new tool results. Clearing unrelated history between tasks is the cheapest optimization available.

The same logic drives lazy loading designs such as progressive skill loading: load instructions on invocation instead of stuffing everything upfront.

Why fuller context performs worse

Anthropic calls this degradation context rot: as context grows, recall of the right detail drops. The engineering goal is therefore the smallest token set that still carries enough signal for a correct answer.

The measurable evidence comes from Chroma in July 2025 across 18 models including Claude Opus 4, Sonnet 4, GPT-4.1, o3, Gemini 2.5, and Qwen3. Accuracy fell as input length grew even when the question and answer stayed identical, which points to length itself as a cause. One near topic distractor paragraph lowered scores, and four lowered them further. On the 306 prompt LongMemEval set, a focused prompt near 300 tokens beat a full dump near 113,000 tokens on all 18 models.

Two caveats keep this honest. Chroma notes its tasks were intentionally simple, so real coding work may degrade more. Claude models tended to abstain when uncertain while GPT models answered confidently but wrong. The same principle motivates selective glossary loading: retrieve narrow cards per task instead of dumping the whole reference.

What compaction keeps and what it loses

Compaction summarizes the conversation near the limit and continues with that summary in a fresh window. Anthropic advises optimizing recall first and precision second, and names tool result clearing as the lightest safe form: drop old tool outputs the agent no longer needs.

The context window docs list exactly what survives:

  • Reloaded intact from disk: system prompt, root CLAUDE.md, rules without path scoping, auto memory, the plan mode plan, a fresh git status snapshot, and the output of any SessionStart hook that matches the compact source.
  • Reloaded with caps: at most the 5 most recent files the agent read or edited (files over 5,000 tokens shrink to a path), plus invoked skill content capped near 5,000 tokens per skill and 25,000 total.
  • Summarized and therefore lossy: the rest of the conversation, including context that hooks added earlier. Path scoped rules and nested CLAUDE.md files reload only when Claude reads or edits their trigger file, so they stay gone until then.

What survives compaction: intact reload, capped reload, and lossy summary

The habits follow directly from that list. Put mandatory constraints in the root CLAUDE.md rather than only in chat. Run /compact with a focus phrase at a task boundary instead of waiting for auto compaction mid task. Avoid quoting a fixed auto compact threshold, since the docs say it varies by model and configuration. To compact earlier, /autocompact 500k sets how full the window gets before the automatic pass runs. Models with a 1 million token window, including Fable models, Sonnet 5 and later, Haiku 5.5, and Opus 4.6 and later, compact the same way at the larger limit.

Why caching makes compaction costly

Prompt caching matches the request prefix. When the start of a request matches the previous one, the cached portion costs less and only the new tail pays full price. Any edit near the front invalidates everything behind it.

Three common actions break the cache: switching models mid session, adding or removing tools or MCP servers when definitions are not deferred, and compaction itself, since the shorter history no longer shares a prefix with the old one.

Whether the cache is still warm determines what compaction costs. While the cache is warm, summarization reads much of the prefix from cache and costs less than the context size suggests. After the TTL expires, the same step processes full history at full price. The documented TTL depends on the request. The main conversation gets 1 hour on a subscription inside quota and 5 minutes on API keys or usage credits, while subagents, compaction, and teammates default to 5 minutes. The promptCacheTtl setting lets you choose 1 hour for the main conversation. Treat third party price quotes as unconfirmed until checked against the official pricing page. The /usage Session block adds a Prompt cache (main) line (Claude Code v2.1.251 and later) with hit rate, misses, expected rebuilds from compaction, and warm or cold state.

/rewind avoids part of that cost by truncating back to a prefix that is already cached. An idle session on the 1 hour TTL may also compact shortly before expiry (idleCompaction, v2.1.290 and later, false turns it off).

Token cuts the Claude Code docs endorse

The official Claude Code cost guide lists habits that need no extra installation. Use /clear between unrelated tasks. Match the model to the job, typically Sonnet for routine code and Opus for tangled architecture. Disable unused MCP servers and prefer CLIs such as gh. Keep CLAUDE.md under about 200 lines and move single workflow instructions into skills that load only on invocation. Push noisy work such as test runs and log reads into subagents so the long output stays in the subagent context and only a summary returns. Write specific prompts that name the file and function so the agent reads less. To see how Claude Code spend compares with Codex on the same benchmark, read Codex vs Claude Code in 2026.

The same cost guide puts average spend near 13 USD per developer per workday, with 90 percent of users under 30 USD per day. Agent team sessions run near 7 times the tokens of a normal session when teammates run in plan mode.

A PreToolUse hook shows the pattern. It rewrites the Bash test command so only failure lines reach the context. The cost guide ships this script, saved as ~/.claude/hooks/filter-test-output.sh:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [
          {
            "type": "command",
            "command": "~/.claude/hooks/filter-test-output.sh"
          }
        ]
      }
    ]
  }
}

Tens of thousands of log tokens collapse to a few hundred kept lines. This is the harness mindset: deterministic checks around the model rather than longer prompts.

Third party cutters and their numbers

Approach What it compresses Reported effect Trust
Caveman proxy Structured input such as logs, CSV, JSON 33.2 percent session input cut per README (54 runs, 18 of 18 correct); per file cuts of 98.5 to 99.1 percent on logs, CSV, and YAML; HTML 9.9 percent worse Medium, README only
Caveman skill Output verbosity Output tokens for 10 dev questions on claude-opus-5-5: 6,983 with no instruction, 4,334 with "Answer concisely", 4,119 with the skill, 2,693 with the terse variant; measured on the 3.1.0 rules and not re-run Medium, style dependent, plus about 1,160 input tokens of rule overhead
LLMLingua Prompt tokens via a small model Up to 20 times on GSM8K with about 1.5 points loss; BBH lost 8.5 points at 5 times and 13.2 at 7 times; clear drops at 25 to 30 times on GSM8K Low for code, QA results do not transfer to character sensitive code

One retired headline deserves emphasis. Version 1.9.1 dropped a 75 percent claim and replaced it with a 65 percent average output cut over a 10 prompt chat benchmark, so quote the 33.2 percent figure with its method attached. Other checks report narrower wins on agent work: Elastic measured a 63.6 percent average reply cut across eight MCP scenarios in its own experiment, and JetBrains saw an 8.5 percent output cut across 86 code tasks, down from about 30 percent in a 10 task pilot. The 9Router dashboard exposes the Caveman option as one of its token savers, and the 9Router guide shows where to find it.

Trade-offs to watch

Compacting right after a long idle forces full price summarization, so compact before the break. Chasing terse output styles distracts from input side wins in logs and JSON, where the measured cuts are largest.

Conclusion

Full history drives cost, longer context weakens answers, compaction keeps only what reloads from disk, and cache breaks make careless compaction expensive. Clear between tasks, pick the right model, keep rule files lean, delegate noisy work, and filter logs with hooks. Use external compressors on structured input, and read every headline claim against its method.

Sources

  1. Anthropic, Effective context engineering for AI agents - context rot framing and recall before precision.
  2. Chroma, Context Rot: How Increasing Input Tokens Impacts LLM Performance - 18 model measurements, distractor and LongMemEval results.
  3. Claude Code docs, Explore the context window - what fills context and what survives compaction.
  4. Claude Code docs, Manage costs effectively - endorsed habits and cost bands.
  5. Claude Code docs, How Claude Code uses prompt caching - prefix matching, TTL, and cache breakers.
  6. JuliusBrussee, Caveman repository - proxy and skill measurements quoted from the README.
  7. Microsoft Research, LLMLingua summary - compression ratios on reasoning and dialog tasks.
  8. LLMLingua paper, arXiv 2310.05736 - method, per dataset results, and high compression degradation.
  9. Caveman v1.9.1 release notes - the retired 75 percent claim and the 65 percent replacement.
  10. Elastic, Elastic-caveman - 63.6 percent average over eight MCP scenarios.
  11. JetBrains, speak to AI agents like cavemen to save tokens - 8.5 percent output cut over 86 tasks.
|Share:

Subscribe to our newsletter

Get the latest articles and updates delivered to your inbox.

pxuanbach
Written by

pxuanbach

Software Engineer at UtilKits. Writing about AI, development, and technology trends.

Related Articles