跳到主内容
智客 ZICQ

技能库 智客分类:Agent 工作流 agent-eval

Agent Eval

基准代码Graph在真实代码库上的检索质量,通过将代理行为与没有代码Graph的vs进行比较. 当用户运行/agent-eval或要求测试,基准,审计,或验证一个代码法版本(本地dev构建或已发布的npm版本)时,使用该语言的repo.

1669 安装量

官方网址:skills.sh

技能介绍

先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。

做什么

将代理行为与无代理行为进行比较,使真实代码库上的基准代码Graph检索质量

何时用

用户运行 / agent- eval 或请求测试、 基准 、 审计 或 验证 编码 版本( 本地 dev 构建 或 已发布的 npm 版本) , 或 请求 测试 语言 repo

代理如何加载

按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:CodeGraph Quality Audit、Prerequisites、Workflow、Notes。 其中含规范建议的小节:分步指令。

文件分析

文件分析:除 SKILL.md 外,正文引用了 scripts/agent-eval/、scripts/agent-eval/audit.sh,属于带资源的技能包,这些文件按需再读。

官方 description(原文)

Benchmark CodeGraph retrieval quality on a real codebase by comparing agent behavior with vs without CodeGraph. Use when the user runs /agent-eval or asks to test, benchmark, audit, or validate a codegraph version (the local dev build or a published npm version) against a language's repo.

CodeGraph Quality AuditPrerequisitesWorkflowNotes

来源分类:skills.sh agent-skill

SKILL.md 与 Agent 调用

官方规范 ↗
name
agent-eval
description
Benchmark CodeGraph retrieval quality on a real codebase by comparing agent behavior with vs without CodeGraph. Use when the user runs /agent-eval or asks to test, benchmark, audit, or validate a codegraph version (the local dev build or a published npm version) against a language's repo.
  1. 发现技能客户端向 Agent 提供名称与描述目录。
  2. 匹配与调用用户指定或任务匹配后,载入 SKILL.md 指令。
  3. 按需加载按步骤读取参考文档、使用脚本与素材。
指令中引用的文件 · 2
  • scripts/agent-eval/
  • scripts/agent-eval/audit.sh

以下路径提取自原文;文件是否齐全请以来源仓库中的完整目录为准。

具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗

安装这个技能

Skills CLI ↗

先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。

该技能引用了附属文件,请从来源获取完整目录;仅复制 SKILL.md 可能缺少依赖。

交给 Agent 安装

复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。

把 Agent Skill「agent-eval」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-650a734f60cedab8-%E5%9F%83%E7%93%A6%E5%B0%94%E6%8E%A2%E5%91%98.html
请存为 .cursor/skills/agent-eval/SKILL.md 或 .claude/skills/agent-eval/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。
该技能还带 scripts/、references/、assets/ 等文件,请从 https://github.com/colbymchenry/codegraph 取完整目录,不要只建一个 SKILL.md。

GitHub 完整包 ↗

终端安装 · Skills CLI

需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。

npx skills add 'https://github.com/colbymchenry/codegraph' --list

npx skills add 'https://github.com/colbymchenry/codegraph' --skill 'agent-eval'

CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。

阅读排版
--- name: agent-eval description: Benchmark CodeGraph retrieval quality on a real codebase by comparing agent behavior with vs without CodeGraph. Use when the user runs /agent-eval or asks to test, benchmark, audit, or validate a codegraph version (the local dev build or a published npm version) against a language's repo. --- # CodeGraph Quality Audit Measures how much CodeGraph helps an agent versus plain grep/read, for a chosen codegraph version on a chosen real-world repo. Drives the harness in `scripts/agent-eval/`. ## Prerequisites - `tmux` 3+, a logged-in `claude` CLI, `node`, `git` (macOS/Linux). - Run from the codegraph repo root. ## Workflow Copy this checklist: ``` - [ ] 1. Pick version (local or npm) - [ ] 2. Pick language - [ ] 3. Pick repo by size - [ ] 4. Pick harness (headless / tmux / both) - [ ] 5. Run audit.sh in the background - [ ] 6. Report results ``` **Step 1 — version.** Ask with `AskUserQuestion`: which codegraph version to test. Offer "Local dev build" and "Latest published"; the free-text "Other" lets the user type a specific version (e.g. `0.7.10`). Map the answer to a VERSION token: - "Local dev build" → `local` - "Latest published" → `latest` - a typed version → that string (e.g. `0.7.10`) **Step 2 — language.** Read `.claude/skills/agent-eval/corpus.json`. Ask with `AskUserQuestion` which language to test, listing the languages that have entries. **Step 3 — repo.** From the chosen language's entries, ask which repo. Label each option with its size and file count, e.g. `excalidraw — Medium (~600 files)`. Each entry carries the `repo` URL and a representative `question`. **Step 4 — harness.** Ask with `AskUserQuestion` which harness to run, and map the answer to a MODE token: - "Headless" → `headless` — `claude -p` with stream-json: exact tokens/cost and a clean tool sequence (2 runs, fast, no TTY). - "Interactive (tmux)" → `tmux` — drives the real Claude TUI in tmux: faithful Explore-subagent behavior, metrics from session logs (2 runs, slower). - "Both" → `all` — headless + interactive (4 runs). **Step 5 — run.** Launch in the background (sets the version, clones if missing, wipes + re-indexes, runs the chosen arms — several minutes): ```bash scripts/agent-eval/audit.sh "" ``` **Step 6 — report.** When the job finishes, read the log and report per arm: - Headless (`parse-run.mjs`): total tool calls, file `Read`s, Grep/Bash, codegraph-tool calls, duration, **total cost**. - Interactive (`parse-session.mjs`): the `VERDICT: codegraph_explore used Nx | Read N | Grep/Bash N` and `TOKENS:` lines. - Both paths also print the three feedback metrics — residual context occupancy, explore sufficiency, allocation efficiency — and a headless A/B ends with a side-by-side `ARM COMPARISON` table. Report that table, and check its contamination row first: `CLI calls that RETURNED output` > 0 means the arm reached codegraph through Bash and its numbers are void. How to read the rest: `docs/benchmarks/agent-eval-feedback-metrics.md`. Lead with cost + tool/Read counts — they are the reliable signals; raw token in/out are confounded by subagent delegation and prompt caching. State whether codegraph reduced effort and whether both arms reached a correct answer. ## Notes - The index is rebuilt every run (`audit.sh` wipes `.codegraph`) — different versions extract differently, so an index must be served by the same binary that built it. - `audit.sh` temporarily mutates the global `codegraph` install for the test, then restores your dev link via `local-install.sh`. - Corpus repos are cloned to `/tmp/codegraph-corpus` (reused if already present). - Add or edit repos in `corpus.json` (fields: `name`, `repo`, `size`, `files`, `question`).

相关技能

Agent 工作流

Skill Creator

创造有效技能指南。 当用户想创造出新的技能(或更新现有的技能),以专业知识,工作流程,或工具集成来扩展克洛德的能力时,应该使用这种技能.

Agent 工作流

Clawdhub

使用ClawdHub CLI搜索,安装,更新并发布从taladhub.com的代理技能. 需要获取苍蝇上的新技能时使用,将安装的技能同步到最新版本或特定版本,或者发布 npm-instainddhub CLI 的新/更新的技能文件夹.

Agent 工作流

Agent Team Orchestration

管弦乐团多代理团队,任务设定周期,交接协议,审查工作流程. 使用时间: (1)建立2+特派员队伍,具有不同专业,(2)确定任务路线和生命周期(收录框_ spec_建设_审查_完成),(3)在特派员之间制定交接协议,(4)建立审查和质量关口,(5)管理特派员之间的交流和文物共享.

Agent 工作流

Superpowers

Spec-first,TDD,子代理驱动的软件开发工作流程. 当:(1)构建任何新功能或应用——触发脑暴_计划_子代理执行回路,(2)调试出一个bug或测试失败——触发系统性的根起过程,(3)用户说"让我们构建","帮助我计划","我想添加X",或"这个被打破",(4)完成一个功…