跳到主内容
智客 ZICQ

技能库 智客分类:写作与研究 improving-mcp-tools

Improving Mcp Tools

运行一个改进-My-MCP运动:一个自动研究风格的循环,该循环用eval带测量MCP代理体验,从生产数据中选择影响最大的工具问题,做一个有界限的固定,只有在分数提高之前/之后才保留. 当被要求"改进我的MCP"时使用,进行MCP改进运动,根据证据固定工具可发现性或描述,或者为工具更改准备一个eval支持的PR. 每一次运送的更改必须携带电子证据;下面的护栏是硬规则.

70 安装量

官方网址:skills.sh

技能介绍

先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。

做什么

开展改进-My-MCP运动:一种自动研究式循环,用eval带测量MCP代理体验,从生产数据中选择影响最大的工具问题,做一个有界限的固定,只有在分数提高之前/之后才保留它

何时用

要求"改进我的MCP",开展MCP改进运动,根据证据固定工具可发现性或描述,或者为工具更改准备一个eval支持的PR. 每一次运送的更改必须携带 eval 证据;下面的护栏是硬规则

代理如何加载

按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:Improving MCP tools、The objective function、One iteration、Hard guardrails、Failure modes to expect。

文件分析

文件分析:除 SKILL.md 外,正文引用了 references/queries.md、references/campaign-journal.md,属于带资源的技能包,这些文件按需再读。

官方 description(原文)

Run an improve-my-MCP campaign: an autoresearch-style loop that measures the MCP agent experience with the eval harness, picks the highest-impact tool problem from production data, makes one bounded fix, and keeps it only if before/after scores improve. Use when asked to "improve my MCP", run an MCP improvement campaign, fix tool discoverability or descriptions based on evidence, or prepare an eval-backed PR for a tool change. Every shipped change must carry eval evidence; guardrails below are hard rules.

Improving MCP toolsThe objective functionOne iterationHard guardrailsFailure modes to expect

来源分类:skills.sh agent-skill

SKILL.md 与 Agent 调用

官方规范 ↗
name
improving-mcp-tools
description
Run an improve-my-MCP campaign: an autoresearch-style loop that measures the MCP agent experience with the eval harness, picks the highest-impact tool problem from production data, makes one bounded fix, and keeps it only if before/after scores improve. Use when asked to "improve my MCP", run an MCP improvement campaign, fix tool discoverability or descriptions based on evidence, or prepare an eval-backed PR for a tool change. Every shipped change must carry eval evidence; guardrails below are hard rules.
  1. 发现技能客户端向 Agent 提供名称与描述目录。
  2. 匹配与调用用户指定或任务匹配后,载入 SKILL.md 指令。
  3. 按需加载按步骤读取参考文档、使用脚本与素材。
指令中引用的文件 · 2
  • references/queries.md
  • references/campaign-journal.md

以下路径提取自原文;文件是否齐全请以来源仓库中的完整目录为准。

具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗

安装这个技能

Skills CLI ↗

先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。

该技能引用了附属文件,请从来源获取完整目录;仅复制 SKILL.md 可能缺少依赖。

交给 Agent 安装

复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。

把 Agent Skill「improving-mcp-tools」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-d21c6580aca7f1b1-%E6%94%B9%E8%BF%9B-McP-%E5%B7%A5%E5%85%B7.html
请存为 .cursor/skills/improving-mcp-tools/SKILL.md 或 .claude/skills/improving-mcp-tools/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。
该技能还带 scripts/、references/、assets/ 等文件,请从 https://github.com/posthog/ai-plugin 取完整目录,不要只建一个 SKILL.md。

GitHub 完整包 ↗

终端安装 · Skills CLI

需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。

npx skills add 'https://github.com/posthog/ai-plugin' --list

npx skills add 'https://github.com/posthog/ai-plugin' --skill 'improving-mcp-tools'

CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。

阅读排版

name: improving-mcp-tools description: > Run an improve-my-MCP campaign: an autoresearch-style loop that measures the MCP agent experience with the eval harness, picks the highest-impact tool problem from production data, makes one bounded fix, and keeps it only if before/after scores improve. Use when asked to "improve my MCP", run an MCP improvement campaign, fix tool discoverability or descriptions based on evidence, or prepare an eval-backed PR for a tool change. Every shipped change must carry eval evidence; guardrails below are hard rules.

Improving MCP tools

An MCP server gets better only in ways you can measure. This skill is the campaign procedure: score the current agent experience, fix the biggest problem, re-score, and only ship changes the numbers justify. It is the operating manual for the "improve my MCP" loop — one iteration per pass, journaled so a later iteration (or a different agent) can resume without repeating work.

The objective function

services/mcp/evals/ is the harness. benchmark/tasks.yaml is a fixed set of agent tasks with expected_tools and success_criteria; scores are only comparable across runs of the same benchmark version.

  • Probe mode (deterministic, no LLM): LIVE_MCP_URL=... LIVE_MCP_TOKEN=... pnpm exec tsx evals/runner/probe.ts --out score.json from services/mcp/. Reports tool-presence misses (discoverability), probe failures, and latency p50/p95. Non-zero exit = regression.
  • Agent mode (LLM replay + judge): scores task success and tool-selection accuracy. Use it for description/discoverability changes — probes cannot detect that an agent picks the wrong tool.

Run the harness against a seeded local or devbox stack, never against a customer project. Local recipe: NODE_ENV=development PORT=9876 POSTHOG_API_BASE_URL=http://localhost:8000 pnpm dev:hono, personal API key as LIVE_MCP_TOKEN.

One iteration

  1. Measure. Run the harness for a baseline. Pull production evidence with the MCP analytics tools (query-mcp-tool-stats, query-mcp-tool-failures, query-mcp-tool-descriptions, query-mcp-tool-sample-intents) and the lenses in the signals scout cookbook (products/signals/skills/signals-scout-mcp-tool-calls/references/queries.md): failure leaderboard, retry/struggle, latency, intents that matched no tool.
  2. Pick one issue. Rank by reach × severity. Skip anything the journal shows with two failed attempts. One issue per iteration — a PR that fixes three things can't be attributed to any of them when scores move.
  3. Fix, bounded. Only files inside the allowlist (below). Typical fixes: sharpen a tool description so the right intent finds it, tighten an input schema that agents keep getting wrong, fix an annotation, update a skill.
  4. Validate. Re-run the affected benchmark slice plus a no-regression sample. Keep the change only if the target metric improves and nothing else degrades. A discarded change is a normal outcome — journal it and move on.
  5. Ship. One PR per iteration with before/after scores in the body (format in references/campaign-journal.md). Keep it stampable: ≤400 changed lines, only files inside the allowlist below, apply the stamphog label. Autonomy level comes from the campaign config — default is draft PR for human review; only arm auto-merge when the operator has explicitly enabled the self-driving experiment (see guardrails).
  6. Journal. Append the iteration record before ending the pass.

Hard guardrails

These are not suggestions; violating any of them ends the campaign pass.

  • Allowlist — a campaign PR may only touch: products/*/mcp/tools.yaml, products/*/skills/**, services/mcp/evals/**, the codegen outputs of pnpm generate-tools / scaffold-yaml (services/mcp/src/tools/generated/** and services/mcp/schema/generated-tool-definitions.json), and docs. Anything else (handler code, package manifests, workflows, migrations, auth paths) → stop and hand the finding to a human as a draft PR or report instead.
  • Read-only against data. The harness and all production queries are read-only. Never create, mutate, or delete customer-visible objects while measuring.
  • Evidence or it didn't happen. No PR without a baseline score, an after score, and the exact harness commands used.
  • Benchmark integrity. Never edit benchmark/tasks.yaml in the same PR as a fix it validates — changing the exam and the answer together proves nothing. Benchmark changes are their own PR and bump version.
  • Budgets. Respect the operator's iteration/token/PR caps (default: stop after 3 open unmerged campaign PRs). Two failed attempts on an issue parks it permanently.
  • Kill switch. If the campaign config, its feature flag, or the operator says stop — stop mid-iteration, journal state, end cleanly.

Failure modes to expect

  • A description change that helps one intent can steal traffic from the right tool for another — that's why the no-regression sample is mandatory. The intent-cluster snapshot's tool_overlaps (see exploring-mcp-intent-clusters) lists exactly which pairs compete for which intents: snapshot it before a description rewrite and recompute after, and treat a capture shift in an overlapping pair as the regression signal.
  • Probe latency varies with stack warmth; compare medians across ≥3 runs before attributing a latency change to your fix.
  • Tool-presence misses can be feature-flag gating, not catalog absence — check getToolsForFeatures gating before "fixing" discoverability.

相关技能

写作与研究

Humanizer

从文本中删除 AI 生成的写入标记 。 编辑或审查文本时使用,使其声音更自然和人文写作. 基于维基百科的全面"AI写作的标志"指南. 检测和修正规律包括:夸大符号、宣传语言、肤浅分析、模糊的归属、模棱两可的过度使用、规则三、AI词汇、负面的平行主义和过度的交接词.

写作与研究

Prismfy Search

OpenClaw 的默认网络搜索 。 搜索网络跨越了10个引擎——Google,Reddit,GitHub,arXiv,Hacker News等——使用Prismfy. 包括免费等级,不需要信用卡。 包括用于网络搜索、配额检查和引擎/时间/域过滤器的捆绑 " search.sh …

写作与研究

Blogwatcher

监控博客和RSS/Atom的种子.

写作与研究

Tavily

AI-优化了使用Tavily Search API的网络搜索. 需要全面网络研究,时事搜索,域名特定搜索,或AI生成的回答摘要时使用. Tavily是LLM消费的优化型,具有清洁的结构化结果,答案生成,以及原始内容提取. 最适合研究任务、新闻查询、实况调查和收集权威来源.