做什么
评估和改进人工智能代理产出的模式和技术。 在以下情况下使用这一技能: - 实施自我约束和反射循环 - 为质量关键一代构建评价-优化管道 - 创建测试驱动的代码完善工作流程 - 设计基于Rumric或LLM-as-judge评价系统 - 在代理输出(代码、报告、分析)中添加迭接改进 - 测量和提高代理响应质量
技能库 智客分类:设计创意 agentic-eval
评估和改进人工智能代理产出的模式和技术。 在以下情况下使用这一技能: - 实施自我约束和反射循环 - 为质量关键一代构建评价-优化管道 - 创建测试驱动的代码完善工作流程 - 设计基于Rumric或LLM-as-judge评价系统 - 在代理输出(代码、报告、分析)中添加迭接改进 - 测量和提高代理响应质量
官方网址:skills.sh
先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。
评估和改进人工智能代理产出的模式和技术。 在以下情况下使用这一技能: - 实施自我约束和反射循环 - 为质量关键一代构建评价-优化管道 - 创建测试驱动的代码完善工作流程 - 设计基于Rumric或LLM-as-judge评价系统 - 在代理输出(代码、报告、分析)中添加迭接改进 - 测量和提高代理响应质量
- 质量关键一代**: 代码、报告、分析要求高度准确
按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:Agentic Evaluation Patterns、Overview、When to Use、Pattern 1: Basic Reflection、Pattern 2: Evaluator-Optimizer、Pattern 3: Code-Specific Reflection。
文件分析:这是一份仅含 SKILL.md 的指令型技能,代理激活后整份正文进入上下文。
Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when: - Implementing self-critique and reflection loops - Building evaluator-optimizer pipelines for quality-critical generation - Creating test-driven code refinement workflows - Designing rubric-based or LLM-as-judge evaluation systems - Adding iterative improvement to agent outputs (code, reports, analysis) - Measuring and improving agent response quality
Agentic Evaluation PatternsOverviewWhen to UsePattern 1: Basic ReflectionPattern 2: Evaluator-OptimizerPattern 3: Code-Specific ReflectionEvaluation StrategiesOutcome-BasedLLM-as-JudgeRubric-BasedBest PracticesQuick Start Checklist
来源分类:skills.sh agent-skill
nameagentic-evaldescription具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗
先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。
复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。
把 Agent Skill「agentic-eval」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-8c434c8b00e40462-%E4%BB%A3%E7%90%86%E5%9F%83%E7%93%A6%E5%B0%94.html 请存为 .cursor/skills/agentic-eval/SKILL.md 或 .claude/skills/agentic-eval/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。
需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。
npx skills add 'https://github.com/github/awesome-copilot' --list
npx skills add 'https://github.com/github/awesome-copilot' --skill 'agentic-eval'
CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。
name: agentic-eval description: | Patterns and techniques for evaluating and improving AI agent outputs. Use this skill when:
Patterns for self-improvement through iterative evaluation and refinement.
Evaluation patterns enable agents to assess and improve their own outputs, moving beyond single-shot generation to iterative refinement loops.
Generate → Evaluate → Critique → Refine → Output
↑ │
└──────────────────────────────┘
Agent evaluates and improves its own output through self-critique.
def reflect_and_refine(task: str, criteria: list[str], max_iterations: int = 3) -> str:
"""Generate with reflection loop."""
output = llm(f"Complete this task:\n{task}")
for i in range(max_iterations):
# Self-critique
critique = llm(f"""
Evaluate this output against criteria: {criteria}
Output: {output}
Rate each: PASS/FAIL with feedback as JSON.
""")
critique_data = json.loads(critique)
all_pass = all(c["status"] == "PASS" for c in critique_data.values())
if all_pass:
return output
# Refine based on critique
failed = {k: v["feedback"] for k, v in critique_data.items() if v["status"] == "FAIL"}
output = llm(f"Improve to address: {failed}\nOriginal: {output}")
return output
Key insight: Use structured JSON output for reliable parsing of critique results.
Separate generation and evaluation into distinct components for clearer responsibilities.
class EvaluatorOptimizer:
def __init__(self, score_threshold: float = 0.8):
self.score_threshold = score_threshold
def generate(self, task: str) -> str:
return llm(f"Complete: {task}")
def evaluate(self, output: str, task: str) -> dict:
return json.loads(llm(f"""
Evaluate output for task: {task}
Output: {output}
Return JSON: {{"overall_score": 0-1, "dimensions": {{"accuracy": ..., "clarity": ...}}}}
"""))
def optimize(self, output: str, feedback: dict) -> str:
return llm(f"Improve based on feedback: {feedback}\nOutput: {output}")
def run(self, task: str, max_iterations: int = 3) -> str:
output = self.generate(task)
for _ in range(max_iterations):
evaluation = self.evaluate(output, task)
if evaluation["overall_score"] >= self.score_threshold:
break
output = self.optimize(output, evaluation)
return output
Test-driven refinement loop for code generation.
class CodeReflector:
def reflect_and_fix(self, spec: str, max_iterations: int = 3) -> str:
code = llm(f"Write Python code for: {spec}")
tests = llm(f"Generate pytest tests for: {spec}\nCode: {code}")
for _ in range(max_iterations):
result = run_tests(code, tests)
if result["success"]:
return code
code = llm(f"Fix error: {result['error']}\nCode: {code}")
return code
Evaluate whether output achieves the expected result.
def evaluate_outcome(task: str, output: str, expected: str) -> str:
return llm(f"Does output achieve expected outcome? Task: {task}, Expected: {expected}, Output: {output}")
Use LLM to compare and rank outputs.
def llm_judge(output_a: str, output_b: str, criteria: str) -> str:
return llm(f"Compare outputs A and B for {criteria}. Which is better and why?")
Score outputs against weighted dimensions.
RUBRIC = {
"accuracy": {"weight": 0.4},
"clarity": {"weight": 0.3},
"completeness": {"weight": 0.3}
}
def evaluate_with_rubric(output: str, rubric: dict) -> float:
scores = json.loads(llm(f"Rate 1-5 for each dimension: {list(rubric.keys())}\nOutput: {output}"))
return sum(scores[d] * rubric[d]["weight"] for d in rubric) / 5
| Practice | Rationale | |----------|-----------| | Clear criteria | Define specific, measurable evaluation criteria upfront | | Iteration limits | Set max iterations (3-5) to prevent infinite loops | | Convergence check | Stop if output score isn't improving between iterations | | Log history | Keep full trajectory for debugging and analysis | | Structured output | Use JSON for reliable parsing of evaluation results |
## Evaluation Implementation Checklist
### Setup
- [ ] Define evaluation criteria/rubric
- [ ] Set score threshold for "good enough"
- [ ] Configure max iterations (default: 3)
### Implementation
- [ ] Implement generate() function
- [ ] Implement evaluate() function with structured output
- [ ] Implement optimize() function
- [ ] Wire up the refinement loop
### Safety
- [ ] Add convergence detection
- [ ] Log all iterations for debugging
- [ ] Handle evaluation parse failures gracefully
设计创意
设计和实施自动化工作流程,以节省时间和规模化操作作为独家. 用于识别重复任务实现自动化,构建跨工具的工作流程,设置触发器和行动,或优化现有自动化. 包括自动化机会识别,工作流程设计,工具选择(Zapier,Make,n8n),测试,和维护. 触发"自动","自动","工作流程自动…
设计创意
UI/UX设计智能及建筑抛光接口实施指导. 当用户要求UI设计,UX流量,信息架构,视觉风格方向,设计系统/托盘,组件规格,副本/显微镜,可访问性,或生成/critique/refine前端UI(HTML/CSS/JS,React,Next.js,Vue,Svelte,Tailw…
设计创意
创建美丽现代UI的专家前端设计指南. 在构建起落架页面,仪表板,或任何用户界面时使用.
设计创意
设计和输出 n8n 工作流程 JSON 有强力触发器, idempotency, 错误处理, 记录, 重试, 以及 人入"一站"审查队列. 当您需要可审计的自动化时使用.