跳到主内容
智客 ZICQ

技能库 智客分类:文档办公 web-content-fetcher

Web Content Fetcher

Extract article content from any URL as clean Markdown. Uses Scrapling script as primary method (with auto fast→stealth fallback), Jina Reader as alternative for simple pages. Preserves headings, links, images, lists, and code blocks. Use this skill whenever the user wants to fetch, read, extract, scrape, or summarize content from a URL — including blog posts, news articles, WeChat articles (微信公众号), documentation pages, or any web page. Also trigger when the user says things like "帮我读一下这篇文章", "抓取这个网页", "提取正文", or "read this page for me".

3031 安装量

官方网址:skills.sh

技能介绍

先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。

做什么

官方 description 用英文写明「Web Content Fetcher」做什么。中文介绍不改写这段原文,能力边界以文件结构与下方原文为准。

何时用

官方 description 未单独写出 Use when。按规范,代理会在用户任务与这段 description 的关键词匹配时激活本技能。

代理如何加载

按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:Web Content Fetcher、Extraction Strategy、Scrapling script、Domain Routing、Script Options、Basic — auto-selects fast or stealth。

文件分析

文件分析:除 SKILL.md 外,正文引用了 scripts/fetch.py,属于带资源的技能包,这些文件按需再读。

Web Content FetcherExtraction StrategyScrapling scriptDomain RoutingScript OptionsBasic — auto-selects fast or stealthForce stealth for known JS-heavy sitesLimit output to 15000 characters (default: 30000)JSON output with metadata (url, mode, selector, content_length)Install DependenciesFailure Rules

来源分类:skills.sh agent-skill

SKILL.md 与 Agent 调用

官方规范 ↗
name
web-content-fetcher
description
Extract article content from any URL as clean Markdown. Uses Scrapling script as primary method (with auto fast→stealth fallback), Jina Reader as alternative for simple pages. Preserves headings, links, images, lists, and code blocks. Use this skill whenever the user wants to fetch, read, extract, scrape, or summarize content from a URL — including blog posts, news articles, WeChat articles (微信公众号), documentation pages, or any web page. Also trigger when the user says things like "帮我读一下这篇文章", "抓取这个网页", "提取正文", or "read this page for me".
  1. 发现技能客户端向 Agent 提供名称与描述目录。
  2. 匹配与调用用户指定或任务匹配后,载入 SKILL.md 指令。
  3. 按需加载按步骤读取参考文档、使用脚本与素材。
指令中引用的文件 · 1
  • scripts/fetch.py

以下路径提取自原文;文件是否齐全请以来源仓库中的完整目录为准。

具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗

安装这个技能

Skills CLI ↗

先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。

该技能引用了附属文件,请从来源获取完整目录;仅复制 SKILL.md 可能缺少依赖。

交给 Agent 安装

复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。

把 Agent Skill「web-content-fetcher」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-857107ba26058fe8-Web-%E5%86%85%E5%AE%B9%E8%8E%B7%E5%8F%96%E5%99%A8.html
请存为 .cursor/skills/web-content-fetcher/SKILL.md 或 .claude/skills/web-content-fetcher/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。
该技能还带 scripts/、references/、assets/ 等文件,请从 https://github.com/shirenchuang/web-content-fetcher 取完整目录,不要只建一个 SKILL.md。

GitHub 完整包 ↗

终端安装 · Skills CLI

需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。

npx skills add 'https://github.com/shirenchuang/web-content-fetcher' --list

npx skills add 'https://github.com/shirenchuang/web-content-fetcher' --skill 'web-content-fetcher'

CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。

阅读排版
--- name: web-content-fetcher description: > Extract article content from any URL as clean Markdown. Uses Scrapling script as primary method (with auto fast→stealth fallback), Jina Reader as alternative for simple pages. Preserves headings, links, images, lists, and code blocks. Use this skill whenever the user wants to fetch, read, extract, scrape, or summarize content from a URL — including blog posts, news articles, WeChat articles (微信公众号), documentation pages, or any web page. Also trigger when the user says things like "帮我读一下这篇文章", "抓取这个网页", "提取正文", or "read this page for me". --- # Web Content Fetcher Given a URL, return its main content as clean Markdown — headings, links, images, lists, code blocks all preserved. ## Extraction Strategy Always try **one method per URL** — don't cascade blindly. Pick the right one upfront. ``` URL │ ├─ 1. Scrapling script (preferred) │ Run fetch.py — check the domain routing table to decide fast vs --stealth. │ Works for most sites. Returns clean Markdown directly. │ └─ 2. Jina Reader (fallback — only if Scrapling fails or dependencies not installed) web_fetch("https://r.jina.ai/") Free tier: 200 req/day. Fast (~1-2s), good Markdown output. Does NOT work for: WeChat (403), some Chinese platforms. ``` ### Scrapling script ```bash python3 /scripts/fetch.py "" [max_chars] [--stealth] ``` `` is the directory where this SKILL.md lives. Resolve it before calling the script. The script has two modes built in: - **Default (fast):** HTTP fetch, ~1-3s, works for most sites - **`--stealth`:** Headless browser, ~5-15s, for JS-rendered or anti-scraping sites When run without `--stealth`, the script automatically falls back to stealth if the fast result has too little content. So you rarely need to specify `--stealth` manually — the only reason to force it is when you already know the site needs it (see routing table), which saves the initial fast attempt. ## Domain Routing Use this table to pick the right mode on the first call: | Domain | Command | Why | |--------|---------|-----| | `mp.weixin.qq.com` | `fetch.py --stealth` | JS-rendered content | | `zhuanlan.zhihu.com` | `fetch.py --stealth` | Anti-scraping + JS | | `juejin.cn` | `fetch.py --stealth` | JS-rendered SPA | | `sspai.com` | `fetch.py ` | Static HTML | | `blog.csdn.net` | `fetch.py ` | Static HTML | | `ruanyifeng.com` | `fetch.py ` | Static blog | | `openai.com` | `fetch.py ` | Static HTML | | `blog.google` | `fetch.py ` | Static HTML | | Everything else | `fetch.py ` | Auto-fallback handles it | ## Script Options ```bash # Basic — auto-selects fast or stealth python3 /scripts/fetch.py "https://sspai.com/post/73145" # Force stealth for known JS-heavy sites python3 /scripts/fetch.py "https://mp.weixin.qq.com/s/xxx" --stealth # Limit output to 15000 characters (default: 30000) python3 /scripts/fetch.py "https://example.com/article" 15000 # JSON output with metadata (url, mode, selector, content_length) python3 /scripts/fetch.py "https://example.com" --json ``` ## Install Dependencies First use only — the script checks and tells you if anything is missing: ```bash pip install scrapling html2text ``` If on system-managed Python (macOS/Linux), add `--break-system-packages` or use a venv. ## Failure Rules - Same URL fails once → give up, tell the user "unable to extract content from this URL" - Do not retry — each failed call wastes context tokens

相关技能

文档办公

Ontology

为结构化的代理内存和可堆肥技能所打入的知识图. 在创建/征服实体(Person, project, Task, Evention, Document)时使用,链接相关对象,强制约束,规划多步动作作为图变,或技能需要共享状态时使用. 触发到"记住","我知道什么","链接X到Y",…

文档办公

Nano Pdf

使用纳米-pdf CLI编辑带有自然语言指令的PDF.

文档办公

Word / DOCX

创建,检查,并编辑有可靠样式的Microsoft Word文档和DOCX文件,编号,跟踪更改,表格,章节,并进行相容性检查. 当 (1) 任务涉及 Word 或 ".docx " 时使用; (2) 文件包括跟踪的更改,评论,字段,表格,模板,或页面布局限制; (3) 文档必须在不…

文档办公

Api Gateway

通过 Maton 网关调用第三方 API, 它为用户已经连接的应用程序注入证书 。 当用户命名一个连接的应用程序和其中的具体行动时使用此技能——读取一个邮箱,查询一个CMM,归档一个问题,更新一个电子表格,通过一个连接的搜索或刮取提供者运行一个查询. 每一次通话都到用户连接的应用…