AgentBaton Hands Off Between Strong and Efficient Models to Cut Agent API Costs by 65.9%

Summary:AgentBaton is a lightweight detector that decides when a capable model should hand a long-horizon agent task over to a cheaper, less capable model. Its main finding is directional: moving from an efficient model to a capable model usually fails to reduce cost, whereas a well-timed capable-to-efficient handoff can preserve or even improve accuracy. Evaluated across 11 model pairs and two benchmarks, AgentBaton cuts API costs by 65.9% on average while achieving accuracy gains of up to 8.8 percenta

---

Core Idea: Split the Task into Two Cost Regimes

AgentBaton targets multi-step, long-horizon agent tasks in which a model repeatedly observes an environment, invokes tools, corrects actions, and maintains task state. Running a flagship model for every turn makes costs scale with trajectory length. Replacing it entirely with a small model can introduce irreversible errors during planning or tool selection.

The paper finds that the useful direction is a handoff from a capable model to an efficient one. The capable model handles the early, high-leverage stages: understanding the objective, decomposing the task, and making critical decisions. Once the remaining work becomes structured and repetitive, an efficient model can continue under a well-timed handoff. The results suggest that one carefully selected handoff can capture most of the benefit without requiring fine-grained routing at every step.

The reverse direction—efficient model to capable model—usually performs worse than capable-only execution while offering limited savings. A weaker early model may corrupt task state, omit constraints, or break the tool-call chain, forcing the stronger model to spend additional tokens recovering from errors. Agent routing must therefore account for recovery cost and state quality, not just per-token pricing.

How the Handoff Detector Works

AgentBaton is a lightweight detector that learns when to switch from the capable model to the efficient model based on recurring patterns in task progress. It makes a state-transition decision within an ongoing task rather than routing isolated requests.

A practical model handoff has three components:

1. State transition: The detector selects the point at which the next inference should use a different model.
2. Context transfer: The capable model's goal, constraints, executed actions, tool results, and remaining work are passed to the efficient model.
3. Continuation: The efficient model completes the remaining steps and must correctly interpret the transferred state.

Passing the entire conversation is expensive because tool results and history continue to grow, inflating input tokens and KV Cache usage. Passing only a compressed summary is cheaper but can discard critical constraints. The available abstract does not specify AgentBaton's context encoder, feature set, training objective, or transfer format, so reproducibility must be assessed from the full paper and any released implementation.

Why Capable-to-Efficient Is the Effective Direction

For a single trajectory, API cost can be expressed as:

[ C = \sum_i (C_{in,i} + C_{out,i}) ]

Input cost grows with context length, while output cost grows with generated tokens. Beyond token pricing, capable models may also incur higher reasoning overhead. A cheaper model can nevertheless become more expensive end to end if it causes more tool retries, longer trajectories, or incorrect environment mutations.

If the capable model handles the first (k) steps and the efficient model handles the remaining (n-k) steps, the approximate cost is:

[ C_{handoff} = k C_{strong} + (n-k) C_{efficient} + C_{transfer} ]

Here, (C_{transfer}) includes serialization, detection, and context migration. The design only pays off when the strong model reduces retry and correction costs by more than the transfer overhead. The reported 65.9% average API-cost reduction suggests substantial gains in the evaluated settings, but it should not yet be generalized to all agent workloads.

Evaluation and the Cost-Accuracy Frontier

The paper evaluates AgentBaton across 11 model pairs, two representative benchmarks, and multiple reasoning-effort settings. It reports:

• a 65.9% average reduction in API cost;
• largely preserved task accuracy;
• accuracy gains of up to 8.8 percentage points over capable-only execution;
• an improved empirical accuracy-cost Pareto frontier.

The key implication is that an efficient model need not be merely a fallback for a stronger model. Under the right task conditions, it may be the better executor for the latter part of a long trajectory.

The abstract does not disclose the complete model-pair list, benchmark composition, sample sizes, statistical significance, per-step token consumption, or end-to-end latency. If the 65.9% figure counts only API charges and excludes migration, detection, retries, and infrastructure, it should be interpreted as model-call cost reduction rather than total task cost reduction. The maximum 8.8-point accuracy gain may also reflect a limited subset of model combinations rather than a general effect.

Engineering Trade-offs

Advantages

• Direct cost savings: Expensive models are reserved for high-risk stages in budget-sensitive search, coding, and automation workflows.
• Low implementation complexity: A single handoff is easier to maintain than per-step routing, voting, and arbitration.
• Preservation of high-end reasoning: Flagship models still handle the most consequential decisions.
• Flexible deployment: The efficient side can use either lower-cost cloud APIs or locally hosted open-weight models.

Costs and Risks

• Context migration can become a bottleneck: Full histories increase token usage and KV Cache pressure; compressed summaries can lose information.
• Errors accumulate: Early mistakes may alter environment state beyond the reach of a smaller model.
• Detector drift: Changes in tool schemas, trajectory length, or model behavior may invalidate learned handoff patterns.
• Only trajectory-level metrics matter: Step accuracy alone cannot represent long-horizon success, retries, or environmental side effects.
• Privacy boundaries: Moving context between cloud providers or from cloud to local inference requires careful handling of secrets, user data, and audit logs.

Deployment Recommendations

A sensible production starting point is a deterministic three-stage pipeline: planning, execution, and verification. Let the capable model perform planning and the initial tool call, let the efficient model execute routine follow-up actions, and use a deterministic checker or inexpensive verifier to validate the result. Once this baseline is stable, a learned detector can replace the fixed boundary, making it easier to distinguish model failures from context-loss and state-corruption problems.

For every handoff, record the task stage, input and output tokens, fees, retry count, tool success rate, context size, and final environment state. Total cost should be measured as:

[ C_{total} = C_{API} + C_{migration} + C_{detection} + C_{retry} + C_{infrastructure} ]

Stress-test boundaries with long contexts, tool failures, prompt injection, model upgrades, and quantized local models. If the efficient model repeatedly makes irreversible changes to the environment, retain capable-model verification rather than relying on a stage transition alone.

Conclusion

AgentBaton offers a more suitable cost-control pattern for long-horizon agents than conventional query-level routing: let a strong model make the high-value decisions, then let an efficient model execute the stable remainder. Its reported 65.9% cost reduction is compelling, but production value ultimately depends on context-transfer quality, recovery behavior, and whether the gains remain stable across models and task distributions.

---

Source: Hugging Face Daily Papers (2026-10-25)

— END —

Tags: #AgentBaton Follow #Model Routing Follow #Multi-Model Collaboration Follow #Agent Cost Optimization Follow #Long-Horizon Tasks Follow