Self Improving Agent

--- name: self-improving-agent description: Use after a failure, user correction, repeated workflow problem, or validated success reveals a reusable lesson. Captures bounded redacted candidates, runs executable behavior evals, and separates validation from application in durable guidance. allowed-tools: Read, Write, Edit, Bash, Grep, Glob --- # Self-Improving Agent Turn evidence from completed work into a small, auditable behavior change. The default result is a candidate or no change—not an automatic rewrite of skills. ## Use This Skill When - A tool or workflow failed in a way that may recur. - The user corrected an assumption, requirement, or operating rule. - The same workaround appeared more than once. - A focused test proved a better reusable method. - The user asks to review or consolidate learning candidates. Do not use it for routine session summaries, raw transcript storage, speculative ideas without evidence, or project facts that belong in project documentation. ## Required Outcome Every run ends in exactly one state: 1. `candidate`: reusable but not yet validated. 2. `validated`: representative evidence supports the lesson, but no owner change is claimed yet. 3. `applied`: the validated lesson was installed in one named durable owner with a change reference. 4. `rejected`: disproved, unsafe, too specific, or obsolete. 5. `superseded` or `rolled_back`: an applied/validated lesson was replaced or reverted. 6. `no-delta`: no reusable behavior change was found. 7. `open-question`: evidence is insufficient and the missing proof is named. An artifact is not proof of improvement. An applied lesson must change future behavior and have a representative check that demonstrates the change. ## Start Packet Before editing durable guidance, state: - Future behavior: what the agent should do differently next time. - Representative task: one concrete scenario that should now succeed. - Evidence: current source, failure output, user correction, or focused test. - Owner: the one skill, instruction file, script, or runtime component that owns it. - Write boundary: files allowed to change and information that must remain local. - Proof: the command, eval, or review that confirms the new behavior. If any item is unknown, capture a candidate and stop before validation or application. ## Lifecycle ### 1. Capture the Signal Prefer facts over interpretation. Record only the minimum reusable summary; do not copy transcripts, tool inputs, credentials, private paths, or customer data. Claude Code failure hooks explicitly enabled with `apb init --hooks` can call: ```bash agent-playbook self-improve ``` Manual corrections or successes use an explicit summary and evidence label: ```bash apb self-improve capture \ --kind correction \ --summary "Verify the current source before relying on cached state" \ --evidence "focused-test" ``` The CLI stores redacted events and deduplicated candidates under `~/.agent-playbook/self-improvement/`. Override the root with `AGENT_PLAYBOOK_DATA_DIR` or `--data-dir`. ### 2. Assess Reusability Keep a candidate only when all are true: - It describes future behavior, not just what happened. - It is useful beyond one private task or repository. - It does not conflict with a current authoritative source. - A narrow owner and a realistic validation path exist. Use `apb behavior inbox` to inspect the prioritized queue. Repeated evidence increases occurrence count; it does not automatically increase truth. Use `apb behavior owners --repo .` for local suggestions, but treat every result as a review candidate rather than an ownership decision. ### 3. Validate Choose the smallest proof that can falsify the candidate, encode it as an executable artifact, and run it with `apb self-improve eval`. See `references/eval-artifact.md` for the schema and safety boundary. | Candidate | Minimum proof | |---|---| | Prompt or workflow rule | Representative prompt plus rubric | | CLI/runtime behavior | Focused automated test | | External integration | Live capability check against current docs/runtime | | Safety rule | Negative test showing the unsafe path is blocked | | Repeated heuristic | Multiple independent episodes or explicit human confirmation | Separate facts, hypotheses, and missing evidence. Structural validation alone does not prove that guidance is semantically current or executable by the host. ### 4. Validate, Apply, or Reject Run the artifact first. A baseline scenario is recommended when the previous behavior can be reproduced safely; at least one candidate scenario is required: ```bash apb self-improve eval cand-123 --artifact behavior-eval.json apb self-improve review cand-123 \ --decision validate \ --reason "baseline reproduced and candidate scenarios passed" \ --eval-result /path/printed/by/the/eval/command.json ``` Validation accepts only a passing CLI-generated eval result for the same candidate. It does not claim runtime behavior changed. Generate a local Behavior Change Proposal before editing the owner: ```bash apb behavior proposal cand-123 \ --owner "skill:self-improving-agent" \ --output behavior-proposal.md ``` The proposal contains the behavior diff intent, eval proof, acceptance criteria, privacy boundary, and rollback plan. It does not edit the owner or create a remote pull request. After changing exactly one durable owner, record the application separately: ```bash apb self-improve review cand-123 \ --decision apply \ --reason "installed after the focused test passed" \ --owner "skill:self-improving-agent" \ --change-ref "commit:abc123" ``` Other decisions: ```bash apb self-improve review cand-123 --decision observe --reason "needs a second episode" apb self-improve review cand-123 --decision reject --reason "project-specific exception" ``` Apply into the narrowest owner: 1. Executable test, script, or validator when behavior can be enforced. 2. The owning skill or its reference when agent judgment is required. 3. Project instructions only for project-wide constraints. 4. A knowledge notebook for durable facts that should be retrieved, not always loaded. Never silently modify repository rules, publish packages, or trigger external actions as a side effect of capture. ### 5. Prove the Loop Run the representative task after application. Report: - candidate id and final state; - evidence used and what remains uncertain; - durable owner changed; - executable eval result and artifact hash; - rollback path. If the new rule does not change the representative behavior, revert or reject it. ## Knowledge Export Export applied rules and open candidates as Markdown for Obsidian or another local knowledge system: ```bash apb self-improve export --output /path/to/vault/Agent/Learning.md ``` The export is a sink, not the source of truth. Candidate and active-rule state remain structured and auditable in the CLI data directory. ## Host Boundary Skills describe judgment; host adapters provide events and actions. Check the current host before claiming support: - Claude Code: deterministic failure hook installed only by explicit `apb init --hooks`. - Codex, Gemini, DeepSeek Harness: skill distribution is supported; learning event wiring depends on each host's current extension API. - Unsupported hooks must remain manual or adapter-specific, never simulated by undocumented behavior. Use `apb conformance` to inspect local-static contracts. A `proven` distribution or hook configuration does not prove host discovery or runtime invocation; those remain `unverified` until an observed host run supplies bounded evidence. See `references/learning-lifecycle.md` for schemas and adapter contracts. Use `evals/cases.json` with `evals/rubric.md` when changing this skill. ## Done Checklist - [ ] Candidate/no-delta decision is explicit. - [ ] Stored text is minimal, redacted, and portable. - [ ] Current authoritative sources were checked when relevant. - [ ] Validation uses a passing executable eval result for the same candidate. - [ ] Application names one durable owner and a concrete change reference. - [ ] Representative behavior was tested after application. - [ ] No private project detail entered public skill assets.