Compacting Reasoning Histories Cuts Token Costs in Long-Horizon Agents
Summary:This work studies a training-free approach for reducing the cost of long reasoning traces. Before pending tool results are revealed, prior reasoning is rewritten into a compact state record, after which the agent continues from either the original or compacted history. Across the evaluated models, accepted compact histories generally lead to shorter subsequent reasoning and fewer repeated verification steps in a coding case study. Software-agent deployments can therefore spend less overall, alth
Core Idea
This work shifts the cost problem of long reasoning traces from asking models to generate less to changing the reasoning history that already exists. Without retraining the model, prior reasoning is rewritten into a compact state record before the next pending tool result is revealed. The model then continues from either the original history or the compacted history.
The intervention is more than truncation. Truncation simply removes unfinished hypotheses, constraints, or intermediate evidence. A state rewrite attempts to retain information needed for future decisions while converting verbose narration into a shorter representation. The paper reports that accepted compact histories generally lead to shorter subsequent reasoning, while a coding case study shows fewer repeated verification steps.
Mechanism
For an agent trajectory, let:
[ H_t = {(o_i, a_i, r_i)}_{i=1}^{t} ]
where (o_i) is an observation, (a_i) is an action, and (r_i) is a tool result. The method constructs:
[ \hat{H}_t = C(H_t) ]
where the compressor (C) should preserve constraints, verified facts, candidate solutions, unresolved work, and tool state rather than merely summarize prose. The model then continues under either (H_t) or (\hat{H}_t), allowing researchers to compare continuation length, repeated actions, and task quality.
The engineering benefits can include shorter input contexts, fewer future output tokens, and fewer unnecessary tool calls. However, the cost model must include the compressor itself. If compaction requires another model call, rule execution, or retrieval, its latency and token usage must be added to the total bill.
Engineering Trade-offs
Aggressive compaction creates information-loss risks. A coding agent may lose failed patches, failure causes, dependency constraints, permission boundaries, unverified solutions, or the correspondence between tool calls and observations. Shorter future reasoning does not necessarily mean that these details were preserved.
The compressor and target model must therefore be evaluated jointly. Models do not share a uniform internal state format: a summary that works for a short mathematical task may remove precisely the debugging details required by a software task.
Latency must also be separated into context-savings latency and rewrite latency. An interactive agent with low concurrency may see little end-to-end improvement, while a long-horizon agent can still save money if fewer continuation tokens and tool calls outweigh the compaction call itself.
Interpreting the Results
The reported evidence is that accepted compact histories usually shorten future reasoning, repeated verification decreases in a coding case, and software-agent deployments can reduce total spending. These results support reasoning history as a useful control variable for future reasoning-token usage.
However, the available summary does not provide the full model suite, compression ratios, acceptance rates, task-level quality changes, compressor token overhead, end-to-end latency distributions, or statistical significance. The result therefore demonstrates a promising direction rather than a universally deployable fixed policy.
A rigorous evaluation should report future output tokens, task success or code pass rates, tool-call counts, compressor overhead, fallback frequency, and error accumulation. Cost per successfully completed task is more informative than average context reduction alone.
Deployment Guidance
The method can be implemented as a runtime compaction layer around the agent loop. Practical safeguards include:
- Trigger compaction after tool observations or before planning rather than at a fixed turn count;
- Preserve goals, hard constraints, verified facts, candidates, failure reasons, and next actions in structured fields;
- Retain limited traceback to the original text for critical state;
- Validate compressed and uncompressed continuations, falling back to the original history when essential constraints disappear;
- Maintain model-specific thresholds and acceptance policies;
- Track tokens, tool calls, latency, success rate, and retries using total cost per successful task as the optimization target.
Enterprise deployments must also enforce sensitive-data minimization and audit controls. Reasoning histories can contain source code, credentials, and internal paths, so compaction must not become a bypass around existing context-access policies.
Significance
The work complements model training, quantization, and KV-cache optimization. It does not alter model parameters or attention computations; instead, it exploits redundancy in an agent's trajectory to reduce the context and generation burden of future turns. Its practical value will depend on preserving capability and safety while reducing total cost, not on shortening traces alone.
— END —Source: Hugging Face Daily Papers (2026-11-02)
Tags: #Reasoning Compaction #Agent Context #Token Cost #Tool Use #Long-Horizon Tasks
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments