跳到主内容
智客 ZICQ

技能库 智客分类:写作与研究 agentic-browser-testing

Agentic Browser Testing

由目标驱动的 E2E 测试中,浏览器代理(Playwright MCP / 计算机使用)读取了自然语言目标,并通过可访问性树来探索应用以断言结果——没有预写脚本. 意向驱动击打脚本时的封面,使代理运行决定力(被绑定的模型,温度0,被种子的数据,被绑定的步骤,明确的成功断言,快照-not-像素),成本/纬度控制,可访问性-树木-第一交互模型,CI gating,并毕业后稳定跑入脚本的Playwright测试. 使用时:"代理浏览器测试","目标驱动浏览器测试","让代理探索应用","自然语言E2E","浏览器代理烟雾测试","Playwright MCP测试". 不用于:写作/维持定型脚本的Playwright测试——即剧作家自定义. 测试你的产品OWN的LLM特性——即AI系统测试. 相關:剧作家自发,AI-系统测试,探索-测试,测试-可靠性,qa-project-context.

765 安装量

官方网址:skills.sh

技能介绍

先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。

做什么

由目标驱动的 E2E 测试中,浏览器代理(Playwright MCP / 计算机使用)读取了自然语言目标,并通过可访问性树来探索应用以断言结果——没有预写脚本. 意图驱动跳动脚本时的封面,使代理运行决定力(被绑定的模型,温度0,被种子的数据,被绑定的步骤,明确的成功断言,快照-不像素),成本/纬度控制,可访问性-树木-第一交互模型,CI Gating,以及从稳定运行升入脚本的Playwright测试

何时用

"代理浏览器测试","目标驱动浏览器测试","让代理探索应用","自然语言E2E","浏览器代理烟雾测试","Playwright MCP测试". 不用于:写作/维持定型脚本的Playwright测试——即剧作家自定义. 测试你的出品

代理如何加载

按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:Quick Route、Discovery Questions、Core Principles、Fit: Intent-Driven vs Scripted、The Interaction Model (accessibility-tree-first)、Determinism: making a run trustworthy in CI。

文件分析

文件分析:除 SKILL.md 外,正文引用了 references/setup.md、references/graduation-and-ci.md,属于带资源的技能包,这些文件按需再读。

官方 description(原文)

Goal-driven E2E testing where a browser agent (Playwright MCP / computer-use) reads a natural-language goal and explores the app via the accessibility tree to assert outcomes — no pre-written script. Covers when intent-driven beats scripted, making agent runs deterministic (pinned model, temperature 0, seeded data, bounded steps, explicit success assertion, snapshot-not-pixel), cost/latency control, the accessibility-tree-first interaction model, CI gating, and graduating a stable run into a scripted Playwright test. Use when: "agentic browser test," "goal-driven browser test," "let an agent explore the app," "natural-language E2E," "browser agent smoke test," "Playwright MCP test." Not for: Writing/maintaining deterministic scripted Playwright tests — that is playwright-automation. Testing your product's OWN LLM features — that is ai-system-testing. Related: playwright-automation, ai-system-testing, exploratory-testing, test-reliability, qa-project-context.

Quick RouteDiscovery QuestionsCore PrinciplesFit: Intent-Driven vs ScriptedThe Interaction Model (accessibility-tree-first)Determinism: making a run trustworthy in CISuccess Assertion: the Oracle (where agents fail silently)Cost and LatencyGraduation and CI GatingMigrating a brittle script to a goal (honest tradeoffs)Anti-Patterns1. Reflexively writing a scripted Playwright test

· 许可:MIT

来源分类:skills.sh agent-skill

SKILL.md 与 Agent 调用

官方规范 ↗
name
agentic-browser-testing
description
Goal-driven E2E testing where a browser agent (Playwright MCP / computer-use) reads a natural-language goal and explores the app via the accessibility tree to assert outcomes — no pre-written script. Covers when intent-driven beats scripted, making agent runs deterministic (pinned model, temperature 0, seeded data, bounded steps, explicit success assertion, snapshot-not-pixel), cost/latency control, the accessibility-tree-first interaction model, CI gating, and graduating a stable run into a scripted Playwright test. Use when: "agentic browser test," "goal-driven browser test," "let an agent explore the app," "natural-language E2E," "browser agent smoke test," "Playwright MCP test." Not for: Writing/maintaining deterministic scripted Playwright tests — that is playwright-automation. Testing your product's OWN LLM features — that is ai-system-testing. Related: playwright-automation, ai-system-testing, exploratory-testing, test-reliability, qa-project-context.
许可
MIT
  1. 发现技能客户端向 Agent 提供名称与描述目录。
  2. 匹配与调用用户指定或任务匹配后,载入 SKILL.md 指令。
  3. 按需加载按步骤读取参考文档、使用脚本与素材。
指令中引用的文件 · 2
  • references/setup.md
  • references/graduation-and-ci.md

以下路径提取自原文;文件是否齐全请以来源仓库中的完整目录为准。

具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗

安装这个技能

Skills CLI ↗

先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。

该技能引用了附属文件,请从来源获取完整目录;仅复制 SKILL.md 可能缺少依赖。

交给 Agent 安装

复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。

把 Agent Skill「agentic-browser-testing」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-02c4106d4277c0ab-%E4%BB%A3%E7%90%86%E6%B5%8F%E8%A7%88%E5%99%A8%E6%B5%8B%E8%AF%95.html
请存为 .cursor/skills/agentic-browser-testing/SKILL.md 或 .claude/skills/agentic-browser-testing/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。
该技能还带 scripts/、references/、assets/ 等文件,请从 https://github.com/petrkindlmann/qa-skills 取完整目录,不要只建一个 SKILL.md。

GitHub 完整包 ↗

终端安装 · Skills CLI

需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。

npx skills add 'https://github.com/petrkindlmann/qa-skills' --list

npx skills add 'https://github.com/petrkindlmann/qa-skills' --skill 'agentic-browser-testing'

CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。

阅读排版
--- name: agentic-browser-testing description: >- Goal-driven E2E testing where a browser agent (Playwright MCP / computer-use) reads a natural-language goal and explores the app via the accessibility tree to assert outcomes — no pre-written script. Covers when intent-driven beats scripted, making agent runs deterministic (pinned model, temperature 0, seeded data, bounded steps, explicit success assertion, snapshot-not-pixel), cost/latency control, the accessibility-tree-first interaction model, CI gating, and graduating a stable run into a scripted Playwright test. Use when: "agentic browser test," "goal-driven browser test," "let an agent explore the app," "natural-language E2E," "browser agent smoke test," "Playwright MCP test." Not for: Writing/maintaining deterministic scripted Playwright tests — that is playwright-automation. Testing your product's OWN LLM features — that is ai-system-testing. Related: playwright-automation, ai-system-testing, exploratory-testing, test-reliability, qa-project-context. license: MIT metadata: author: kindlmann version: "1.0" category: ai-qa --- A scripted Playwright test breaks the moment a button moves or a class renames; writing one for a dashboard that changes weekly is a maintenance treadmill. This skill stands up a goal-driven browser agent instead: it reads a natural-language goal, explores the app via the accessibility tree (Playwright MCP `browser_snapshot`), and asserts the outcome against an explicit oracle. The failure mode it prevents is the one that makes teams distrust agents — an agent that reports "success" while stuck on the login page because nothing forced it to prove where it landed. You leave with a deterministic, CI-gated agent run and a graduation path to a durable scripted test once the flow stabilizes. ## Quick Route | Situation | Go to | |-----------|-------| | Stand up a goal-driven run from scratch | Discovery + `references/setup.md` | | Decide agentic vs scripted for a given flow | Fit: Intent-Driven vs Scripted | | Agent passes one run, fails the next | Determinism | | "How does it click without screenshots?" | Interaction Model | | Runs are slow / burning tokens | Cost and Latency | | Agent reports false success | Success Assertion (the Oracle) | | Flow is stable — make it permanent | Graduation → `references/graduation-and-ci.md` | | Block a merge on the goal | CI Gating → `references/graduation-and-ci.md` | | Canvas / no accessibility tree | Canvas Fallback → `references/graduation-and-ci.md` | ## Discovery Questions First, check `.agents/qa-project-context.md` in the project root and skip anything it already answers (stack, environments, seed/reset tooling, model access). 1. **Which flow, and how often does its UI change?** Fast-changing/experimental UI favors intent-driven; a stable critical path (login) favors scripted. This decides the whole approach. 2. **Is there a seeded fixture and a way to reset state?** Determinism is impossible without seeded data and a per-run reset. If neither exists, that is step zero. 3. **Can you deep-link past auth to a seeded entry point?** Re-driving login every run is the biggest avoidable cost; a seeded entry URL scopes the goal and cuts steps. 4. **What is the unambiguous success oracle?** Specific account text, a `/dashboard` URL, an order number — plus a forbidden state. "No error" is not an oracle. 5. **Does the target render to canvas / WebGL?** No accessibility tree means snapshot-first won't work; plan the vision fallback or instrument the canvas with ARIA. 6. **Which model and budget?** Pin a model id and a step budget up front; tier cheap steps to Haiku 4.5 / Sonnet 4.6 and reserve Opus 4.8 for genuinely ambiguous flows. --- ## Core Principles 1. **Intent, not instructions — but only where churn earns it.** The agent reads a goal and finds its own path through the accessibility tree, so it survives a moved button or renamed class that would break a selector. That resilience costs 2-5x the time and money of a scripted run, so spend it on fast-changing UI and hard-to-locate flows, not on stable critical paths. 2. **An agent run is untrustworthy until it is deterministic.** Same goal, same seeded app must produce the same verdict. That requires temperature 0, a pinned model id, seeded data with a reset, a bounded step budget, and an explicit pass/fail assertion. Without these you have a coin flip, not a test. 3. **The oracle lives outside the agent.** Never let the LLM self-grade "looks good." Success is a checkable assertion against the final `browser_snapshot` — specific expected text, a URL, AND a forbidden-state negative check — evaluated by your harness, not the model. 4. **Accessibility tree first, pixels last.** `browser_snapshot` returns roles, refs, and accessible names (~200-400 tokens) and is deterministic and cheap. Screenshots, pixel coordinates, vision, and OCR are a scoped last resort for canvas only, never the default. 5. **Graduation is the goal, not perpetual agent runs.** Once a flow is stable, promote the run to a durable scripted `tests/*.spec.ts` with role-based locators. An agent that has been green for two weeks should become a fast, free regression test — keep the agent for exploration, not for guarding a settled path. --- ## Fit: Intent-Driven vs Scripted The decision is per-flow, not per-project. Run `risk-based-testing` first if you need the risk map; this table is the routing rule once you have it. | Flow characteristic | Use | Why | |---------------------|-----|-----| | Stable, high-frequency critical path (login, payment) | **Scripted + pinned** (`playwright-automation`) | Runs every PR; must be fast, free, and deterministic. No upside to re-exploring it. | | Fast-changing / experimental UI (a dashboard that churns weekly, a redesign in flight) | **Agentic / intent-driven** | Selectors would break constantly; a goal survives layout churn. | | Hard-to-locate flow you can't reliably select | **Agentic** | The agent finds the control by role/name instead of you reverse-engineering a selector. | | Exploratory smoke / "does the happy path still work at all" | **Agentic** | One NL goal covers a lot of ground without a maintained script. | | Anything in CI that must never falsely pass | Scripted, OR agentic **with a hard oracle** | Non-determinism is a false-pass risk you must actively cap. | **The rule, stated plainly:** keep stable critical paths scripted and pinned; point intent-driven agents at fast-changing UI and exploratory smoke. Do not move everything to the agent — it is slower, costlier, and non-deterministic, and not every test should be agentic. --- ## The Interaction Model (accessibility-tree-first) Playwright MCP is **not** computer-use with screenshots and pixel coordinates. It is accessibility-tree-first: 1. `browser_navigate` to the seeded entry URL. 2. `browser_snapshot` returns the **accessibility tree** — each interactive element as a `role`, a stable `ref`, and its `accessible name` (from ARIA/labels). ~200-400 tokens. 3. The agent picks an element by `ref` and calls `browser_click` or `browser_type`. 4. `browser_wait_for` waits on text appearing/disappearing — never a fixed sleep. 5. Re-`browser_snapshot` after the DOM changes; assert against that tree. Why not screenshots: the snapshot is **token-efficient** (thousands of tokens cheaper than an image), **deterministic** (text refs, not fuzzy pixel matching), and needs no vision model or OCR. Feeding screenshots as the primary input makes the run slower, pricier, and flakier. `browser_take_screenshot` is for human evidence only, never as the assertion input. See `references/setup.md` for the MCP registration, the full tool table, and the goal prompt. --- ## Determinism: making a run trustworthy in CI A run that passes once and fails the next with no app change is not yet a test. The fix is never "just retry" or bumping temperature for "smarter" exploration — that adds variance. Pin the variables instead: | Lever | Setting | |-------|---------| | Model | **Pinned model id** (e.g. `claude-haiku-4-5-20251001`), never `latest` | | Sampling | **temperature 0** — no creative wandering in CI | | Data | **Seeded fixture + reset/seed the database** before every run | | Scope | **Bounded step budget** (`maxSteps`), e.g. 18 — exceeding it FAILS, never auto-retries | | Oracle | **Explicit pass/fail verdict** asserted against the snapshot | | Evidence | Assert on the **accessibility tree**, never a screenshot diff | Avoid: `temperature: 0.7` or `1` for exploration, retry-until-pass loops, `waitForTimeout` sleeps, and screenshot-based assertions. Each one hides flakiness rather than removing it. Full harness config in `references/setup.md`. --- ## Success Assertion: the Oracle (where agents fail silently) This is the sharpest failure mode: the agent reports success while stuck on the login page, because "page loaded / no error / looks good" was accepted as success and the LLM was allowed to self-grade. Force an explicit oracle the harness checks — never the agent. For the goal *"sign in as an existing user and confirm the dashboard shows the right account name"*: ```text SUCCESS (all must hold — assert against the final browser_snapshot): - URL matches /dashboard - Snapshot contains the specific expected account name text, e.g. "Acme Corp — Jane R." NEGATIVE / forbidden state (fail fast if any is true): - Still on a URL matching /login → FAIL - Snapshot contains role="alert" with "invalid credentials" → FAIL VERDICT: harness emits {"passed": true|false}; the LLM does not decide. ``` The positive checks (specific account name + `/dashboard` URL) prove where it landed; the **negative check** (must NOT be on the login page) is what kills the false pass. "No error," "didn't crash," "screenshot looks correct," and "trust the agent" are not success criteria. --- ## Cost and Latency Agent runs are 2-5x slower and pricier than scripted tests — a step is an LLM round-trip, the dominant cost. Cut spend without losing coverage by going *smaller*, not *bigger*: - **Step budget** — keep `maxSteps` low and enforced; fewer round-trips, less drift. - **Model tiering** — Haiku 4.5 / Sonnet 4.6 for cheap navigation steps; reserve Opus 4.8 for genuinely ambiguous exploration. Don't run the biggest model on every step. - **Prompt caching** — cache the static system prompt, tool schemas, and goal; they repeat every run. - **Scope via a seeded entry point** — one narrow goal per run, deep-linked past login instead of re-driving it each time. - **Snapshot over screenshots** — the a11y snapshot is ~200-400 tokens; a full-page screenshot is thousands. Default to snapshot. Backwards moves to reject: "use a bigger model / Opus 4.8 for every step," "raise the step limit," "screenshot every step," and running with no budget at all. See `references/setup.md`. --- ## Graduation and CI Gating Promote a stabilized goal into a durable scripted test, and gate merges on the verdict. Both are detailed in `references/graduation-and-ci.md`; the essentials: - **Graduate** with **Playwright Test Agents** (planner / generator / healer, shipped in Playwright **v1.56.0**). `npx playwright init-agents --loop=claude`. The **planner** writes a Markdown test plan to `specs/.md`; the **generator** turns it into `tests/.spec.ts` with **role-based locators** (`getByRole`, `getByLabel`, `getByText`) verified against the live DOM; the **healer** repairs broken locators. This is the promotion path — not "keep running it as an agent," not recorded clicks, not `page.locator('xpath=...')`, not data-testid-only. - **Gate CI** so a failed goal exits **non-zero** and emits a **machine-readable** verdict (`{"passed": true|false}` in `result.json`); the GitHub Actions job parses the boolean and `exit 1`s on false. State is seeded/ephemeral and reset per run, with a step budget and a timeout cap. Never `continue-on-error: true`, never "always exit 0," never a prose verdict a human reads. - **Canvas with no accessibility tree:** prefer instrumenting the canvas with ARIA; as a scoped last resort enable `--caps=vision` to unlock `browser_mouse_click_xy` for that flow only. `browser_snapshot` will not work on a raw canvas, but don't make coordinates the default and don't abandon agentic testing. --- ## Migrating a brittle script to a goal (honest tradeoffs) Converting an 80-line script that re-types login and walks 6 hardcoded steps into a single NL **goal** with an **explicit success assertion** is a real win for a churning flow — but state the downsides honestly: - **Non-determinism / false-pass risk** — the run could pass falsely; that's why the hard oracle and the negative check are non-negotiable. - **Cost/latency** — 2-5x slower; bound it with a **step budget** and a seeded entry point. - **Not every test should be agentic** — keep stable paths scripted, and plan to **graduate** this one back to a scripted test once it stabilizes. Reject the over-promise: it is **not** "strictly better with no downsides," do **not** "migrate everything," and never drop the assertions to make it pass. --- ## Anti-Patterns ### 1. Reflexively writing a scripted Playwright test "Browser test" pattern-matches to codegen, so the default is `page.goto` / `page.locator` / `await expect(page...)` / hunting `data-testid`. That misses the entire point. A goal-driven agent reads NL intent and explores via `browser_snapshot` — no pre-written selectors. ### 2. "Use the agent for everything" Over-selling the new toy. The agent is 2-5x slower and non-deterministic. Stable critical paths (login) stay scripted and pinned; intent-driven wins on fast-changing UI. Never "always use the agent" or "agents replace all scripted tests." ### 3. Fixing flakiness with retries or higher temperature "Just retry" and bumping temperature for "smarter" exploration both add variance. The real levers are temperature 0, a pinned model, seeded data, a bounded step budget, and an explicit verdict. ### 4. Assuming computer-use = screenshots + pixel coordinates Playwright MCP is accessibility-tree-first. Defaulting to vision, screenshots, OCR, or `mouse_click_xy` is slower, costlier, and flakier. Pixels are a canvas-only last resort. ### 5. "No error = success" (the false pass) Accepting "page loaded / didn't crash / looks good" and letting the LLM self-grade is exactly why the agent reports success while stuck on login. Require specific expected text, a URL, and a forbidden-state negative check, evaluated by the harness. ### 6. Bigger model / more steps to go faster Backwards. Opus 4.8 on every step and raising `maxSteps` raise cost and latency without buying reliability. Smaller models, tighter budgets, caching, and tighter scope are the fix. ### 7. Running an agent forever instead of graduating A goal green for two weeks on a now-stable flow should become a scripted `tests/*.spec.ts` via Playwright Test Agents. "Keep running it as an agent," recording clicks, and xpath locators are all wrong promotions. ### 8. Prose verdict in CI Returning a paragraph for a human to read, or `continue-on-error: true`, lets a failed goal merge. The run must exit non-zero on failure with a machine-readable boolean. --- ## Done When - Goal prompt exists as natural-language intent (no `page.locator` / `page.goto` / `data-testid` in the goal) with a START seeded entry URL. - An explicit success oracle is defined: specific expected text AND a URL check AND a forbidden-state negative check, asserted against `browser_snapshot` — not a screenshot. - Run config pins a model id, sets `temperature: 0`, a `maxSteps` budget, and a seed; no `waitForTimeout`, no retry-until-pass. - Interaction is snapshot-first: `browser_navigate` / `browser_snapshot` / `browser_click` / `browser_type` / `browser_wait_for`; screenshots used only for evidence. - The runner emits `result.json` with `{"passed": true|false}` and `exit 1`s on false; the CI job gates the merge on the boolean (no `continue-on-error`, no `always exit 0`). - CI seeds/resets ephemeral state per run and enforces a step budget and a timeout cap. - A graduation trigger is recorded (e.g. "green for 2 weeks → run `init-agents`, generate `tests/.spec.ts` with `getByRole` locators"). - If any target is canvas/WebGL, the vision fallback (`--caps=vision` + `browser_mouse_click_xy`) is scoped to that flow only, or the canvas is instrumented with ARIA. --- ## Related Skills - **playwright-automation** — Writing and maintaining deterministic scripted Playwright tests and Page Objects. Go there to author the durable test; this skill graduates an agent run into one. - **ai-system-testing** — Testing your product's OWN LLM/AI features (prompt regression, model output quality). This skill tests any app *using* an agent; it does not test your AI feature. - **exploratory-testing** — Human SBTM exploration and bug hunting. The agentic smoke goal is the automated cousin; use exploratory-testing for charter-driven manual sessions. - **test-reliability** — Self-healing locators and quarantine for *scripted* flaky tests at runtime. Complements the determinism levers here once a test has graduated. - **qa-project-context** — The universal dependency; supplies stack, environments, seed/reset tooling, and model access that every question above depends on. ## Reference Files (in `references/`) - **setup.md** — Playwright MCP registration (`.mcp.json`), the snapshot tool table, the natural-language goal prompt with success/negative assertions, the determinism harness config (pinned model, temperature 0, maxSteps, seed, prompt cache), and cost/latency levers. - **graduation-and-ci.md** — Playwright Test Agents promotion pipeline (planner → `specs/*.md`, generator → `tests/*.spec.ts` with role-based locators, healer), the GitHub Actions gating workflow with a machine-readable boolean verdict, and the canvas `--caps=vision` fallback.

相关技能

写作与研究

Humanizer

从文本中删除 AI 生成的写入标记 。 编辑或审查文本时使用,使其声音更自然和人文写作. 基于维基百科的全面"AI写作的标志"指南. 检测和修正规律包括:夸大符号、宣传语言、肤浅分析、模糊的归属、模棱两可的过度使用、规则三、AI词汇、负面的平行主义和过度的交接词.

写作与研究

Prismfy Search

OpenClaw 的默认网络搜索 。 搜索网络跨越了10个引擎——Google,Reddit,GitHub,arXiv,Hacker News等——使用Prismfy. 包括免费等级,不需要信用卡。 包括用于网络搜索、配额检查和引擎/时间/域过滤器的捆绑 " search.sh …

写作与研究

Blogwatcher

监控博客和RSS/Atom的种子.

写作与研究

Tavily

AI-优化了使用Tavily Search API的网络搜索. 需要全面网络研究,时事搜索,域名特定搜索,或AI生成的回答摘要时使用. Tavily是LLM消费的优化型,具有清洁的结构化结果,答案生成,以及原始内容提取. 最适合研究任务、新闻查询、实况调查和收集权威来源.