SIM Continues Agent Harness Optimization with a 2.36-Point Coding Gain
Summary:Continual Recursive Self-Improvement (CRSI) asks an optimizer to carry an evolved harness across compatible benchmarks or policy models. The paper's Self-Improvement Memory (SIM) sits on the optimizer side: it groups edits by recurring harness problem and stores each attempt's rationale, predicted and observed outcomes, and evaluation context. New attempts extend this history as optimization continues. Across four domains, SIM produced higher final scores than harness-only and root-cause-only c
Scope and Core Idea
This paper studies how a harness optimizer can continue its work, rather than how a foundation model updates its weights. When the benchmark or the policy model changes, retaining only the final evolved harness discards evidence about why earlier edits were made and how they performed. The paper defines this setting as Continual Recursive Self-Improvement (CRSI) and introduces Self-Improvement Memory (SIM).
In the context of the paper, a harness is an evolving execution scaffold around a task-solving policy. It may include prompts, tool orchestration, verification, and retry logic, although the supplied abstract does not enumerate its components. The important boundary is that SIM belongs to the optimizer side rather than being injected directly into the evaluated policy's prompt or context.
Technical Mechanism
The memory state described by the abstract can be abstracted as:
M_t = {e_1, e_2, ..., e_n}
Each edit record e_i contains at least four categories of information:
- Problem grouping: Edits are grouped around recurring harness problems, allowing experience to be retrieved by failure mode instead of being appended as an undifferentiated timeline.
- Rationale: The optimizer records why an edit was made, preserving a causal trace of the optimization decision.
- Predicted and observed outcomes: Both expectations and measured results are stored so that later attempts can revise a strategy rather than repeatedly rediscover the same change.
- Evaluation context: The benchmark, policy model, and evaluation conditions are retained to determine whether prior experience remains applicable.
New attempts extend this history, making SIM closer to procedural memory for harness optimization than to a one-shot example selector. The abstract does not disclose whether grouping uses rules, embeddings, or language-model judgment. It also omits Top-k retrieval, memory compaction, and conflict-resolution policies, all of which matter for reproduction.
The three continuation baselines clarify the intended contribution:
| Continuation strategy | Inherited content | Main limitation | |---|---|---| | Harness-only | Final evolved scaffold | No rationale, outcome history, or applicability context | | Root-cause-only | Root cause or edit rationale | No predicted outcome, observed outcome, or full evaluation conditions | | SIM | Problem, rationale, prediction, observation, and context | Must manage stale memory, retrieval errors, and storage growth |
SIM's practical value is the transition from recording what changed to recording why it changed, what was expected, what happened, and under which evaluation conditions. This is more suitable for transfer across benchmarks and models than copying a final prompt or workflow that may be overfit to an earlier benchmark.
Engineering Trade-offs and Empirical Results
The experiments cover four domains, including cross-dataset continuation and pairwise cross-model continuation. SIM achieves higher final scores than harness-only and root-cause-only continuation in these comparisons. In the coding experiments, ten independent source runs are evaluated; paired continuations with SIM improve final scores by 2.36 points on average over harness-only continuation, with p=0.003.
The paired design controls for some source-run variability, while the cross-dataset and cross-model comparisons provide stronger evidence than a single fixed setting. Nevertheless, 2.36 points is an absolute difference rather than a relative percentage gain. Because the supplied material omits baseline scores, score scale, and confidence intervals, the practical effect size remains unclear. A p-value of 0.003 indicates that the measured difference is unlikely under the paper's statistical assumptions, but it does not establish robustness across every model or task family. The abstract also omits comparison counts, multiple-testing treatment, random seeds, policy models, benchmark composition, and failure cases.
SIM also introduces deployment costs:
- Memory growth: Retaining full evaluation context for every attempt makes storage grow approximately linearly with the number of trials. Injecting the entire history into the optimizer context adds a further token-cost burden.
- Stale experience: Changes to benchmark versions, tool interfaces, model weights, or scorers can invalidate earlier records. Ignoring evaluation context increases the risk of inappropriate reuse.
- Misattribution: Incorrect problem grouping or causal attribution can cause an accidental gain to be mistaken for a transferable rule.
- Retrieval error: Excessive reuse narrows exploration, while no retrieval degenerates into ordinary trial-and-error.
- Evaluation dependence: Predicted outcomes are useful only when tied to reliable observed evaluations; scorer drift can contaminate the history.
- Additional overhead: The task policy may incur no extra inference cost, but the optimizer must store, retrieve, and maintain extra evaluation records, potentially increasing total optimization cost.
Deployment Guidance
- Keep memory in the optimizer control layer: Do not inject the full history into the task policy by default. Let the optimizer decide whether to reuse, revise, or reject an experience, then pass only the confirmed minimal change to the execution harness.
- Use an auditable schema: In addition to problem grouping, rationale, predictions, and observations, record model version, benchmark version, tool-interface version, prompt hash, random seed, evaluation time, and cost.
- Gate reuse on contextual compatibility: Permit direct reuse only when the benchmark family, model generation, toolchain, and scoring protocol satisfy predefined compatibility rules. Otherwise run a small smoke test first.
- Adopt shadow evaluation and gradual promotion: Replay candidate changes on historical problems before evaluating them on the current task set. Promote a harness only after it passes regression and cost thresholds, while retaining an immediate rollback path.
- Retain failed attempts: Failed edits and prediction errors are necessary for calibrating the optimizer and should not be discarded, since a success-only history systematically overestimates the value of similar changes.
- Evaluate total cost per successful task: Track optimizer calls, tokens, wall-clock time, incorrect reuse rate, stale-memory rate, and rollback count in addition to final quality.
- Reproduce with three matched controls: Compare harness-only, root-cause-only, and SIM using identical source runs, models, budgets, and seeds. Report paired differences, confidence intervals, and consistency across domains.
Conclusion
SIM contributes an auditable optimization-memory layer for CRSI: historical edits are no longer just a final scaffold, but records with problems, rationales, predictions, observations, and evaluation context. The current evidence supports its potential for cross-benchmark and cross-model continuation, but does not demonstrate autonomous recursive improvement without external evaluation constraints. The most defensible engineering role for SIM is therefore an experimental memory and change-control layer for harness optimizers.
— END —Source: Hugging Face Daily Papers (2026-11-13)
Tags: #Agentic Self-Improvement #Harness Optimization #Continual Memory #Cross-Benchmark Transfer #Agent Evaluation
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments