Coca-Cola Compresses Tool Outputs into Continuous Latents for LLM Agents
Summary:Researchers from THU-Agent propose Coca-Cola, a continuous latent context compression framework for tool-using LLM agents. Rather than truncating, pruning, or summarizing raw tool observations in discrete text space, Coca-Cola encodes them into latent tokens aligned with the agent's input embedding space. Its Action-Preserving Training objective aligns the agent's next-action distributions under full and compressed observations, encouraging the compressed representation to retain decision-releva
Core Idea: Moving Tool Observations Beyond the Text Context
Conventional LLM agents append search results, web pages, or API responses directly to their context. Tool calls are discrete, but their outputs are often long, noisy, and only weakly relevant to the next decision. Once inserted into subsequent turns, this material increases inference cost and can dilute attention to task-critical evidence.
Coca-Cola changes the representation itself. It encodes each tool observation into a small set of continuous latent tokens aligned with the target LLM's input embedding space. The policy no longer processes the raw observation token by token; instead, it conditions its next action on a compressed representation of the evidence.
Unlike token pruning or abstractive summarization, which must make discrete choices within a text vocabulary, a continuous latent representation can retain fine-grained semantic signals without generating readable text. It also avoids the latency and token cost of asking an LLM to summarize tool output before continuing the trajectory.
Mechanism: Aligning Action Distributions
Coca-Cola does not merely optimize reconstruction or textual similarity. Its Action-Preserving Training (APT) objective aligns the agent's next-action distributions under full and compressed observations:
$$P_{full}(a_t \mid s_t, o_t) \approx P_{compressed}(a_t \mid s_t, z_t)$$
Here, $o_t$ denotes the original tool observation and $z_t$ represents its compressed latent form. Because the training signal is derived from action distributions, the compressor is encouraged to preserve the evidence needed for the next decision rather than every detail of the source text.
Architecturally, the compressor acts as an adapter between tool outputs and the policy model. It produces vectors compatible with the target model's embedding space, allowing the agent to continue without re-encoding the compressed content through another LLM call. The trade-off is model coupling: the compressor must learn or adapt to the target model's embedding dimensionality, normalization, and positional-input conventions.
Engineering Value and Deployment Constraints
Experiments cover web-search agents across four QA benchmarks and two model families. The paper reports a 70.01%–89.24% reduction in token consumption, a 24.45%–55.97% reduction in end-to-end runtime, and consistently improved task performance.
The gains have two likely components. First, shorter agent inputs reduce prefill computation and KV-cache pressure. Second, compressed observations may suppress irrelevant search noise that would otherwise distract the policy. The latter is particularly important: a good compression system must preserve details that alter decisions—such as negations, dates, quantities, and source provenance—not merely remove low-frequency text.
Production adoption still involves three main costs:
- Compression latency: The compressor must run after each tool response, and its throughput must be included in end-to-end measurements.
- Representation adaptation: Latent tokens must match the policy model's embedding interface, potentially requiring retraining when the backbone changes.
- Auditability: Unlike textual summaries, latent states are difficult to inspect directly. Production systems need probes, confidence monitoring, and fallback paths to the original observations.
Evaluating the Results
The fact that performance improves alongside efficiency is meaningful. It suggests that some removed tokens were not merely redundant but actively harmful to decision quality. An action-aligned objective can therefore optimize compression and agent reliability together.
The evidence, however, is still concentrated on web-search QA. These tasks have relatively clear success criteria and constrained tool spaces, so they do not establish universal superiority for code execution, browser automation, or long-horizon database workflows. Independent evaluation should test unseen tools and domains, fixed latent-token budgets with very long observations, comparison against token pruning and summarization at equal cost, and robustness to erroneous or malicious tool output. It should also measure whether compression errors accumulate over long trajectories.
Practical Deployment Guidance
Teams operating search agents over large web or API responses should begin with environments that have stable action spaces. Record action distributions and final task success under the full context, then gradually substitute compressed observations while comparing action consistency, answer quality, and end-to-end latency. If the system requires auditable evidence, retain raw tool outputs in external storage and allow low-confidence cases to query the original source.
The primary metric should be cost per successful task, not token compression alone. Track model tokens, tool calls, compressor latency, KV-cache usage, retry rates, and evidence lookups together to ensure that lower context usage does not conceal a larger retry or verification cost.
— END —Source: Hugging Face Daily Papers (2026-11-04)
Tags: #Coca-Cola #Agent Context Compression #Continuous Latent Tokens #Tool Use #Action-Preserving Training
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments