跳到主内容
智客 ZICQ

技能库 智客分类:媒体内容 automate-this

Automate This

分析一个手动过程的屏幕记录,并制作目标明确的工作自动化脚本. 从视频文件中取出帧和音频解说,重建分步工作流程,并使用已经安装在用户机上的工具,提出多个复杂层次的自动化.

2350 安装量

官方网址:skills.sh

技能介绍

先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。

做什么

分析一个手动过程的屏幕记录,并制作目标明确的工作自动化脚本. 从视频文件中取出帧和音频解说,重建分步工作流程,并使用已经安装在用户机上的工具,提出多个复杂层次的自动化.

何时用

官方 description 未单独写出 Use when。按规范,代理会在用户任务与这段 description 的关键词匹配时激活本技能。

代理如何加载

按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:Automate This、Prerequisites Check、Phase 1: Extract Content from the Recording、Frame Extraction、Audio Extraction and Transcription、Use whichever whisper binary is available。

文件分析

文件分析:这是一份仅含 SKILL.md 的指令型技能,代理激活后整份正文进入上下文。

官方 description(原文)

Analyze a screen recording of a manual process and produce targeted, working automation scripts. Extracts frames and audio narration from video files, reconstructs the step-by-step workflow, and proposes automation at multiple complexity levels using tools already installed on the user machine.

Automate ThisPrerequisites CheckPhase 1: Extract Content from the RecordingFrame ExtractionAudio Extraction and TranscriptionUse whichever whisper binary is availablePhase 2: Reconstruct the ProcessPhase 3: Environment FingerprintPhase 4: Propose AutomationTier StructureProposal FormatTier [N]: [Name]

来源分类:skills.sh agent-skill

SKILL.md 与 Agent 调用

官方规范 ↗
name
automate-this
description
Analyze a screen recording of a manual process and produce targeted, working automation scripts. Extracts frames and audio narration from video files, reconstructs the step-by-step workflow, and proposes automation at multiple complexity levels using tools already installed on the user machine.
  1. 发现技能客户端向 Agent 提供名称与描述目录。
  2. 匹配与调用用户指定或任务匹配后,载入 SKILL.md 指令。
  3. 按需加载按步骤读取参考文档、使用脚本与素材。

具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗

安装这个技能

Skills CLI ↗

先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。

交给 Agent 安装

复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。

把 Agent Skill「automate-this」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-aedad67163e7010f-%E8%87%AA%E5%8A%A8%E5%8C%96-%E8%BF%99%E4%B8%AA.html
请存为 .cursor/skills/automate-this/SKILL.md 或 .claude/skills/automate-this/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。

GitHub 完整包 ↗

终端安装 · Skills CLI

需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。

npx skills add 'https://github.com/github/awesome-copilot' --list

npx skills add 'https://github.com/github/awesome-copilot' --skill 'automate-this'

CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。

阅读排版
--- name: automate-this description: 'Analyze a screen recording of a manual process and produce targeted, working automation scripts. Extracts frames and audio narration from video files, reconstructs the step-by-step workflow, and proposes automation at multiple complexity levels using tools already installed on the user machine.' --- # Automate This Analyze a screen recording of a manual process and build working automation for it. The user records themselves doing something repetitive or tedious, hands you the video file, and you figure out what they're doing, why, and how to script it away. ## Prerequisites Check Before analyzing any recording, verify the required tools are available. Run these checks silently and only surface problems: ```bash command -v ffmpeg >/dev/null 2>&1 && ffmpeg -version 2>/dev/null | head -1 || echo "NO_FFMPEG" command -v whisper >/dev/null 2>&1 || command -v whisper-cpp >/dev/null 2>&1 || echo "NO_WHISPER" ``` - **ffmpeg is required.** If missing, tell the user: `brew install ffmpeg` (macOS) or the equivalent for their OS. - **Whisper is optional.** Only needed if the recording has narration. If missing AND the recording has an audio track, suggest: `pip install openai-whisper` or `brew install whisper-cpp`. If the user declines, proceed with visual analysis only. ## Phase 1: Extract Content from the Recording Given a video file path (typically on `~/Desktop/`), extract both visual frames and audio: ### Frame Extraction Extract frames at one frame every 2 seconds. This balances coverage with context window limits. ```bash WORK_DIR=$(mktemp -d "${TMPDIR:-/tmp}/automate-this-XXXXXX") chmod 700 "$WORK_DIR" mkdir -p "$WORK_DIR/frames" ffmpeg -y -i "" -vf "fps=0.5" -q:v 2 -loglevel warning "$WORK_DIR/frames/frame_%04d.jpg" ls "$WORK_DIR/frames/" | wc -l ``` Use `$WORK_DIR` for all subsequent temp file paths in the session. The per-run directory with mode 0700 ensures extracted frames are only readable by the current user. If the recording is longer than 5 minutes (more than 150 frames), increase the interval to one frame every 4 seconds to stay within context limits. Tell the user you're sampling less frequently for longer recordings. ### Audio Extraction and Transcription Check if the video has an audio track: ```bash ffprobe -i "" -show_streams -select_streams a -loglevel error | head -5 ``` If audio exists: ```bash ffmpeg -y -i "" -ac 1 -ar 16000 -loglevel warning "$WORK_DIR/audio.wav" # Use whichever whisper binary is available if command -v whisper >/dev/null 2>&1; then whisper "$WORK_DIR/audio.wav" --model small --language en --output_format txt --output_dir "$WORK_DIR/" cat "$WORK_DIR/audio.txt" elif command -v whisper-cpp >/dev/null 2>&1; then whisper-cpp -m "$(brew --prefix 2>/dev/null)/share/whisper-cpp/models/ggml-small.bin" -l en -f "$WORK_DIR/audio.wav" -otxt -of "$WORK_DIR/audio" cat "$WORK_DIR/audio.txt" else echo "NO_WHISPER" fi ``` If neither whisper binary is available and the recording has audio, inform the user they're missing narration context and ask if they want to install Whisper (`pip install openai-whisper` or `brew install whisper-cpp`) or proceed with visual-only analysis. ## Phase 2: Reconstruct the Process Analyze the extracted frames (and transcript, if available) to build a structured understanding of what the user did. Work through the frames sequentially and identify: 1. **Applications used** — Which apps appear in the recording? (browser, terminal, Finder, mail client, spreadsheet, IDE, etc.) 2. **Sequence of actions** — What did the user do, in order? Click-by-click, step-by-step. 3. **Data flow** — What information moved between steps? (copied text, downloaded files, form inputs, etc.) 4. **Decision points** — Were there moments where the user paused, checked something, or made a choice? 5. **Repetition patterns** — Did the user do the same thing multiple times with different inputs? 6. **Pain points** — Where did the process look slow, error-prone, or tedious? The narration often reveals this directly ("I hate this part," "this always takes forever," "I have to do this for every single one"). Present this reconstruction to the user as a numbered step list and ask them to confirm it's accurate before proposing automation. This is critical — a wrong understanding leads to useless automation. Format: ``` Here's what I see you doing in this recording: 1. Open Chrome and navigate to [specific URL] 2. Log in with credentials 3. Click through to the reporting dashboard 4. Download a CSV export 5. Open the CSV in Excel 6. Filter rows where column B is "pending" 7. Copy those rows into a new spreadsheet 8. Email the new spreadsheet to [recipient] You repeated steps 3-8 three times for different report types. [If narration was present]: You mentioned that the export step is the slowest part and that you do this every Monday morning. Does this match what you were doing? Anything I got wrong or missed? ``` Do NOT proceed to Phase 3 until the user confirms the reconstruction is accurate. ## Phase 3: Environment Fingerprint Before proposing automation, understand what the user actually has to work with. Run these checks: ```bash echo "=== OS ===" && uname -a echo "=== Shell ===" && echo $SHELL echo "=== Python ===" && { command -v python3 && python3 --version 2>&1; } || echo "not installed" echo "=== Node ===" && { command -v node && node --version 2>&1; } || echo "not installed" echo "=== Homebrew ===" && { command -v brew && echo "installed"; } || echo "not installed" echo "=== Common Tools ===" && for cmd in curl jq playwright selenium osascript automator crontab; do command -v $cmd >/dev/null 2>&1 && echo "$cmd: yes" || echo "$cmd: no"; done ``` Use this to constrain proposals to tools the user already has. Never propose automation that requires installing five new things unless the simpler path genuinely doesn't work. ## Phase 4: Propose Automation Based on the reconstructed process and the user's environment, propose automation at up to three tiers. Not every process needs three tiers — use judgment. ### Tier Structure **Tier 1 — Quick Win (under 5 minutes to set up)** The smallest useful automation. A shell alias, a one-liner, a keyboard shortcut, an AppleScript snippet. Automates the single most painful step, not the whole process. **Tier 2 — Script (under 30 minutes to set up)** A standalone script (bash, Python, or Node — whichever the user has) that automates the full process end-to-end. Handles common errors. Can be run manually when needed. **Tier 3 — Full Automation (under 2 hours to set up)** The script from Tier 2, plus: scheduled execution (cron, launchd, or GitHub Actions), logging, error notifications, and any necessary integration scaffolding (API keys, auth tokens, etc.). ### Proposal Format For each tier, provide: ``` ## Tier [N]: [Name] **What it automates:** [Which steps from the reconstruction] **What stays manual:** [Which steps still need a human] **Time savings:** [Estimated time saved per run, based on the recording length and repetition count] **Prerequisites:** [Anything needed that isn't already installed — ideally nothing] **How it works:** [2-3 sentence plain-English explanation] **The code:** [Complete, working, commented code — not pseudocode] **How to test it:** [Exact steps to verify it works, starting with a dry run if possible] **How to undo:** [How to reverse any changes if something goes wrong] ``` ### Application-Specific Automation Strategies Use these strategies based on which applications appear in the recording: **Browser-based workflows:** - First choice: Check if the website has a public API. API calls are 10x more reliable than browser automation. Search for API documentation. - Second choice: `curl` or `wget` for simple HTTP requests with known endpoints. - Third choice: Playwright or Selenium for workflows that require clicking through UI. Prefer Playwright — it's faster and less flaky. - Look for patterns: if the user is downloading the same report from a dashboard repeatedly, it's almost certainly available via API or direct URL with query parameters. **Spreadsheet and data workflows:** - Python with pandas for data filtering, transformation, and aggregation. - If the user is doing simple column operations in Excel, a 5-line Python script replaces the entire manual process. - `csvkit` for quick command-line CSV manipulation without writing code. - If the output needs to stay in Excel format, use openpyxl. **Email workflows:** - macOS: `osascript` can control Mail.app to send emails with attachments. - Cross-platform: Python `smtplib` for sending, `imaplib` for reading. - If the email follows a template, generate the body from a template file with variable substitution. **File management workflows:** - Shell scripts for move/copy/rename patterns. - `find` + `xargs` for batch operations. - `fswatch` or `watchman` for triggered-on-change automation. - If the user is organizing files into folders by date or type, that's a 3-line shell script. **Terminal/CLI workflows:** - Shell aliases for frequently typed commands. - Shell functions for multi-step sequences. - Makefiles for project-specific task sets. - If the user ran the same command with different arguments, that's a loop. **macOS-specific workflows:** - AppleScript/JXA for controlling native apps (Mail, Calendar, Finder, Preview, etc.). - Shortcuts.app for simple multi-app workflows that don't need code. - `automator` for file-based workflows. - `launchd` plist files for scheduled tasks (prefer over cron on macOS). **Cross-application workflows (data moves between apps):** - Identify the data transfer points. Each transfer is an automation opportunity. - Clipboard-based transfers in the recording suggest the apps don't talk to each other — look for APIs, file-based handoffs, or direct integrations instead. - If the user copies from App A and pastes into App B, the automation should read from A's data source and write to B's input format directly. ### Making Proposals Targeted Apply these principles to every proposal: 1. **Automate the bottleneck first.** The narration and timing in the recording reveal which step is actually painful. A 30-second automation of the worst step beats a 2-hour automation of the whole process. 2. **Match the user's skill level.** If the recording shows someone comfortable in a terminal, propose shell scripts. If it shows someone navigating GUIs, propose something with a simple trigger (double-click a script, run a Shortcut, or type one command). 3. **Estimate real time savings.** Count the recording duration and multiply by how often they do it. "This recording is 4 minutes. You said you do this daily. That's 17 hours per year. Tier 1 cuts it to 30 seconds each time — you get 16 hours back." 4. **Handle the 80% case.** The first version of the automation should cover the common path perfectly. Edge cases can be handled in Tier 3 or flagged for manual intervention. 5. **Preserve human checkpoints.** If the recording shows the user reviewing or approving something mid-process, keep that as a manual step. Don't automate judgment calls. 6. **Propose dry runs.** Every script should have a mode where it shows what it *would* do without doing it. `--dry-run` flags, preview output, or confirmation prompts before destructive actions. 7. **Account for auth and secrets.** If the process involves logging in or using credentials, never hardcode them. Use environment variables, keychain access (macOS `security` command), or prompt for them at runtime. 8. **Consider failure modes.** What happens if the website is down? If the file doesn't exist? If the format changes? Good proposals mention this and handle it. ## Phase 5: Build and Test When the user picks a tier: 1. Write the complete automation code to a file (suggest a sensible location — the user's project directory if one exists, or `~/Desktop/` otherwise). 2. Walk through a dry run or test with the user watching. 3. If the test works, show how to run it for real. 4. If it fails, diagnose and fix — don't give up after one attempt. ## Cleanup After analysis is complete (regardless of outcome), clean up extracted frames and audio: ```bash rm -rf "$WORK_DIR" ``` Tell the user you're cleaning up temporary files so they know nothing is left behind.

相关技能

媒体内容

Nano Banana Pro

以纳米·香蕉Pro(Gemini 3 Pro Image)来生成/编辑图像. 用于创建/修改请求,包括编辑。 支持文本到图像+图像到图像; 1K/2K/4K; 使用 -- input-image.

媒体内容

Youtube Watcher

从YouTube视频中获取并读取文字记录. 需要汇总视频时使用,回答有关视频内容的问题,或者从中提取信息.

媒体内容

Atxp

Access ATXP支付API工具用于网页搜索,AI图像生成,音乐创建,视频生成,X/Twitter搜索,电子邮件,代理账户管理. 当用户需要实时网络搜索时使用,AI生成的媒体(图像,音乐,视频),X/Twitter搜索,发送/接收电子邮件,或创建和资助代理账户. 需要通过“ …

媒体内容

Youtube

YouTube Data API与管理的OAuth的集成. 搜索视频,管理播放列表,访问频道数据,并与评论互动. 用户想与YouTube互动时使用此技能. 对于其他第三方应用,使用api-gateway技能(https://clawhub.ai/byungkyu/api-gate…