ZICQ
中 Log in / Sign up
Newsroom Agentic #GUARD #Recursive Self-Improvement #Agent Harness #Directional Regret #Continual Learning

GUARD Stabilizes Recursive Self-Improvement with Directional Regret

Avatar of Mr.Xu

By Mr.Xu Compiled & Reviewed by Editorial

Published:

中文阅读 (Chinese) English Version
Co-reading Select any sentence in the article to highlight, annotate, or copy a quote card. Join discussion ↓

Summary:GUARD targets recursive self-improvement in agents that keep a frozen base model but continuously edit an external harness containing prompts, skills, tool rules, and memory. It converts tasks into pre-execution self-checks, uses directional regret to distinguish newly broken behaviors from unresolved ones, ranks parallel edit candidates through critic-free group-relative fitness, and accepts an update only when a Soft-HURT gate finds that its gains outweigh its regressions. The paper reports g


Core Design

GUARD limits recursive self-improvement to an editable harness composed of prompts, skills, tool rules, and memory while keeping the base-model weights frozen. Its target is not to train a stronger model, but to let an agent revise its operating rules continuously without sacrificing capabilities it has already mastered.

Conventional self-editing pipelines often judge candidate patches using reflection text or a single scalar score. Both forms of feedback conflate a regression introduced by the current patch with a problem that was already unresolved. GUARD uses directional regret to separate those cases: regressions should trigger rollback or repair, while persistent failures remain eligible for further optimization.

Mechanism

1. Pre-Execution Checks

GUARD decomposes each task into pre-execution self-checks and then uses those checks to evaluate complete trajectories. Compared with asking only whether the final task succeeded, this creates a finer-grained view of tool-rule compliance, retained skills, memory conflicts, and memory growth.

The checks effectively act as local reward functions in a reference-free environment. The abstract does not disclose who generates them, how they are revised, or how checker errors are handled. GUARD therefore reduces dependence on an expensive external grader, but does not eliminate the need for validation.

2. Directional Regret

After each update, GUARD compares behavior under the new rules with previously observed behavior. Rules that are already mastered are pinned, newly broken behaviors receive greater update priority, and unresolved failures are no longer collapsed into the same feedback signal as newly introduced regressions.

This distinction is central to safe accumulation. A system optimizing only aggregate success can repair one task by damaging several established abilities. Directional regret makes regression an explicit constraint rather than something visible only after averaging away the damage.

3. Critic-Free Group-Relative Selection

GUARD generates multiple edits in parallel and ranks them through relative fitness within the candidate group, without invoking a separate critic model. This can lower evaluation cost and latency while avoiding some shared biases between the executor and an external judge.

The approach nevertheless depends on meaningful candidate diversity. If every candidate makes the same mistake, relative ranking can still consistently select an incorrect patch. Candidate count, sampling diversity, and task coverage therefore remain important engineering controls.

4. Soft-HURT Gate

Soft-HURT accepts a candidate only when its gains outweigh its regressions. Specialized fixes are also scoped to their intended behaviors, reducing the risk that a newly added rule over-triggers in merely similar situations. The abstract states that the gate guarantees non-regression on its gated buffer.

At an engineering level, the acceptance rule can be approximated as:

Accept(e) = 1[G(e) > λR(e)]

Here, G(e) denotes gain, R(e) denotes regression, and λ controls risk tolerance. This is an explanatory abstraction rather than the paper's disclosed formula. More importantly, the non-regression guarantee applies only to the gated buffer; it is not a global guarantee across future tasks, tool states, or distribution-shift inputs.

5. Single-Pass Deployment

After harness evolution, deployment uses one frozen harness in a single pass. It does not repeatedly self-evaluate, reflect, and rewrite rules for every request. The expensive optimization loop is moved offline, leaving production latency closer to that of a conventional prompt-and-tool agent.

Engineering Trade-offs

Freezing the model avoids backpropagation, gradient updates, and model-copy management. It also makes every harness revision inspectable, versionable, and reversible. Because learning occurs outside the parameters, skill forgetting has a smaller and more controllable blast radius.

The cost is shifted to the offline loop. Every round generates, executes, and compares multiple candidates, potentially consuming substantial model calls and tokens. A reduction in deployment tokens does not automatically imply a proportional reduction in total lifecycle cost. Operators must count candidate generation, self-checking, regression evaluation, and final serving together.

Harness growth creates another constraint. Accumulating rules and memories expands the context window, increases time to first token, and can dilute critical instructions. GUARD's gate reduces memory pollution, but production deployments still need separate token budgets, time-to-live policies, deduplication, and conflict merging for rules, skills, and memories.

Frozen weights also impose a ceiling. Knowledge that cannot be expressed through tools, prompts, or external memory cannot be acquired through harness evolution. If the bottleneck is the base model's reasoning ability rather than orchestration, returns will diminish quickly.

How to Read the Evaluation

The abstract covers memory, skill evolution, continual task streams, and domain reasoning, claiming better performance, less forgetting and memory pollution, and lower deployment-token usage. It does not disclose benchmark names, absolute scores, variance, sample sizes, base models, context limits, or tool-call costs. It is therefore impossible to determine how much of the gain comes from the mechanism rather than a larger inference budget or task-specific tuning.

The critical ablations are removal of pre-execution checks, directional regret, group-relative selection, and Soft-HURT; transfer to unseen task streams; whether token savings hide a larger harness or more offline calls; and whether a checker sharing the executor's model exhibits correlated errors.

Deployment Recommendations

  • Treat the harness as an auditable artifact. Version prompts, skills, rules, and memories separately, and record their scope and token footprint so a change can be localized or rolled back.
  • Maintain both a golden regression suite and a hidden holdout set. The former checks non-regression, while the latter detects benchmark-specific overfitting.
  • Separate deterministic checks from model judgments. Program rules are suitable for tool arguments, permissions, and state consistency; model-based checks should handle semantic constraints.
  • Gate releases on offline candidate cost, final harness size, p50/p95 latency, task success, severe-regression rate, and tokens per successful task—not average score alone.
  • Start automatic updates on low-risk, replayable tasks. Any rule that triggers external tools, permissions, or writes should require human approval or an environment-level verifier.
  • GUARD's reference-free design works well as a first filtering layer. High-risk deployments still need an independent oracle, manual audits, or verifiable environmental rewards, because a shared checker-executor failure can be amplified by the closed loop.

Source: ArXiv AI (cs.AI) (2026-11-19)

— END —

Tags: #GUARD #Recursive Self-Improvement #Agent Harness #Directional Regret #Continual Learning

Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.

What's your take on this story?
🚀 Explore & Submit Works

Community Comments

Loading live comments and annotations…