跳到主内容
智客 ZICQ

技能库 智客分类:文档办公 bright-data-best-practices

Bright Data Best Practices

建立生产准备的 " 明亮数据 " 集成系统,并纳入最佳做法。 用于开发者使用编码助理(Claude Code, Cursor等)的参考文档,用于实施网页刮取,搜索,浏览器自动化,以及结构化的数据提取. 封面为Web Unlocker API,SERP API,Web Scraper API,并浏览器API (Scraping浏览器).

1715 安装量

官方网址:skills.sh

技能介绍

先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。

做什么

建立生产准备的 " 明亮数据 " 集成系统,并纳入最佳做法。 用于开发者使用编码助理(Claude Code, Cursor等)的参考文档,用于实施网页刮取,搜索,浏览器自动化,以及结构化的数据提取. 封面为Web Unlocker API,SERP API,Web Scraper API,并浏览器API (Scraping浏览器).

何时用

官方 description 未单独写出 Use when。按规范,代理会在用户任务与这段 description 的关键词匹配时激活本技能。

代理如何加载

按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:CLI Setup Reference、Bright Data APIs、Choosing the Right API、Authentication Pattern (All APIs)、Web Unlocker API、Key Parameters。

文件分析

文件分析:除 SKILL.md 外,正文引用了 references/cli-setup.md、references/web-unlocker.md、references/serp-api.md、references/web-scraper-api.md、references/browser-api.md,属于带资源的技能包,这些文件按需再读。

官方 description(原文)

Build production-ready Bright Data integrations with best practices baked in. Reference documentation for developers using coding assistants (Claude Code, Cursor, etc.) to implement web scraping, search, browser automation, and structured data extraction. Covers Web Unlocker API, SERP API, Web Scraper API, and Browser API (Scraping Browser).

CLI Setup ReferenceBright Data APIsChoosing the Right APIAuthentication Pattern (All APIs)Web Unlocker APIKey ParametersQuick PatternsGet markdown (best for LLM input)Geo-targeted requestScreenshot for debuggingAsync for bulk processingSERP API

来源分类:skills.sh agent-skill

SKILL.md 与 Agent 调用

官方规范 ↗
name
bright-data-best-practices
description
Build production-ready Bright Data integrations with best practices baked in. Reference documentation for developers using coding assistants (Claude Code, Cursor, etc.) to implement web scraping, search, browser automation, and structured data extraction. Covers Web Unlocker API, SERP API, Web Scraper API, and Browser API (Scraping Browser).
  1. 发现技能客户端向 Agent 提供名称与描述目录。
  2. 匹配与调用用户指定或任务匹配后,载入 SKILL.md 指令。
  3. 按需加载按步骤读取参考文档、使用脚本与素材。
指令中引用的文件 · 5
  • references/cli-setup.md
  • references/web-unlocker.md
  • references/serp-api.md
  • references/web-scraper-api.md
  • references/browser-api.md

以下路径提取自原文;文件是否齐全请以来源仓库中的完整目录为准。

具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗

安装这个技能

Skills CLI ↗

先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。

该技能引用了附属文件,请从来源获取完整目录;仅复制 SKILL.md 可能缺少依赖。

交给 Agent 安装

复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。

把 Agent Skill「bright-data-best-practices」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-04b8e24ab69a5ae9-%E4%BA%AE%E7%9B%B8%E6%95%B0%E6%8D%AE%E6%9C%80%E4%BD%B3%E5%81%9A%E6%B3%95.html
请存为 .cursor/skills/bright-data-best-practices/SKILL.md 或 .claude/skills/bright-data-best-practices/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。
该技能还带 scripts/、references/、assets/ 等文件,请从 https://github.com/brightdata/skills 取完整目录,不要只建一个 SKILL.md。

GitHub 完整包 ↗

终端安装 · Skills CLI

需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。

npx skills add 'https://github.com/brightdata/skills' --list

npx skills add 'https://github.com/brightdata/skills' --skill 'bright-data-best-practices'

CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。

阅读排版

name: bright-data-best-practices description: "Build production-ready Bright Data integrations with best practices baked in. Reference documentation for developers using coding assistants (Claude Code, Cursor, etc.) to implement web scraping, search, browser automation, and structured data extraction. Covers Web Unlocker API, SERP API, Web Scraper API, and Browser API (Scraping Browser)." user-invocable: false

CLI Setup Reference

Install, authentication, and troubleshooting for the Bright Data CLI (bdata) are documented in a single canonical place:

→ references/cli-setup.md

Consult it before any task that shells out to bdata.

Bright Data APIs

Bright Data provides infrastructure for web data extraction at scale. Four primary APIs cover different use cases — always pick the most specific tool for the job.

Choosing the Right API

| Use Case | API | Why | |----------|-----|-----| | Scrape any webpage by URL (no interaction) | Web Unlocker | HTTP-based, auto-bypasses bot detection, cheapest | | Google / Bing / Yandex search results | SERP API | Specialized for SERP extraction, returns structured data | | Structured data from Amazon, LinkedIn, Instagram, TikTok, etc. | Web Scraper API | Pre-built scrapers, no parsing needed | | Click, scroll, fill forms, run JS, intercept XHR | Browser API | Full browser automation | | Puppeteer / Playwright / Selenium automation | Browser API | Connects via CDP/WebDriver | | Route your own HTTP client through a raw proxy (DC/ISP/Residential/Mobile) | Proxy networks | When you need direct proxy access with your own request logic instead of a managed API — see the proxy.md skill |

Authentication Pattern (All APIs)

All APIs share the same authentication model. The env vars below apply to direct REST API integrations — if you are using the bdata CLI, bdata login handles all of these automatically (see references/cli-setup.md).

export BRIGHTDATA_API_KEY="your-api-key"         # From Control Panel > Account Settings
export BRIGHTDATA_UNLOCKER_ZONE="zone-name"       # Web Unlocker zone name
export BRIGHTDATA_SERP_ZONE="serp-zone-name"      # SERP API zone name
export BROWSER_AUTH="brd-customer-ID-zone-NAME:PASSWORD"  # Browser API credentials

REST API authentication header for Web Unlocker and SERP API:

Authorization: Bearer YOUR_API_KEY

Web Unlocker API

HTTP-based scraping proxy. Best for simple page fetches without browser interaction.

Endpoint: POST https://api.brightdata.com/request

import requests

response = requests.post(
    "https://api.brightdata.com/request",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "zone": "YOUR_ZONE_NAME",
        "url": "https://example.com/product/123",
        "format": "raw"
    }
)
html = response.text

Key Parameters

| Parameter | Type | Description | |-----------|------|-------------| | zone | string | Zone name (required) | | url | string | Target URL with http:// or https:// (required) | | format | string | "raw" (HTML) or "json" (structured wrapper) (required) | | method | string | HTTP verb, default "GET" | | country | string | 2-letter ISO for geo-targeting (e.g., "us", "de") | | data_format | string | Transform: "markdown" or "screenshot" | | async | boolean | true for async mode |

Quick Patterns

# Get markdown (best for LLM input)
response = requests.post(
    "https://api.brightdata.com/request",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={"zone": ZONE, "url": url, "format": "raw", "data_format": "markdown"}
)

# Geo-targeted request
json={"zone": ZONE, "url": url, "format": "raw", "country": "de"}

# Screenshot for debugging
json={"zone": ZONE, "url": url, "format": "raw", "data_format": "screenshot"}

# Async for bulk processing
json={"zone": ZONE, "url": url, "format": "raw", "async": True}

Critical rule: Never use Web Unlocker with Puppeteer, Playwright, Selenium, or anti-detect browsers. Use Browser API instead.

See references/web-unlocker.md for complete reference including proxy interface, special headers, async flow, features, and billing.


SERP API

Structured search engine result extraction for Google, Bing, Yandex, DuckDuckGo.

Endpoint: POST https://api.brightdata.com/request (same as Web Unlocker)

response = requests.post(
    "https://api.brightdata.com/request",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "zone": "YOUR_SERP_ZONE",
        "url": "https://www.google.com/search?q=python+web+scraping&brd_json=1&gl=us&hl=en",
        "format": "raw"
    }
)
data = response.json()
for result in data.get("organic", []):
    print(result["rank"], result["title"], result["link"])

Essential Google URL Parameters

| Parameter | Description | Example | |-----------|-------------|---------| | q | Search query | q=python+web+scraping | | brd_json | Parsed JSON output | brd_json=1 (always use for data pipelines) | | gl | Country for search | gl=us | | hl | Language | hl=en | | start | Pagination offset | start=10 (page 2), start=20 (page 3) | | tbm | Search type | tbm=nws (news), tbm=isch (images), tbm=vid (videos) | | brd_mobile | Device | brd_mobile=1 (mobile), brd_mobile=ios | | brd_browser | Browser | brd_browser=chrome | | brd_ai_overview | Trigger AI Overview | brd_ai_overview=2 | | uule | Encoded geo location | for precise location targeting |

Note: num parameter is deprecated as of September 2025. Use start for pagination.

Parsed JSON Response Structure

{
  "organic": [{"rank": 1, "global_rank": 1, "title": "...", "link": "...", "description": "..."}],
  "paid": [],
  "people_also_ask": [],
  "knowledge_graph": {},
  "related_searches": [],
  "general": {"results_cnt": 1240000000, "query": "..."}
}

Bing Key Parameters

| Parameter | Description | |-----------|-------------| | q | Search query | | setLang | Language (prefer 4-letter: en-US) | | cc | Country code | | first | Pagination (increment by 10: 1, 11, 21...) | | safesearch | off, moderate, strict | | brd_mobile | Device type |

Async for Bulk SERP

# Submit
response = requests.post(
    "https://api.brightdata.com/request",
    params={"async": "1"},
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={"zone": SERP_ZONE, "url": "https://www.google.com/search?q=test&brd_json=1", "format": "raw"}
)
response_id = response.headers.get("x-response-id")

# Retrieve (retrieve calls are NOT billed)
result = requests.get(
    "https://api.brightdata.com/serp/get_result",
    params={"response_id": response_id},
    headers={"Authorization": f"Bearer {API_KEY}"}
)

Billing: Pay per 1,000 successful requests only. Async retrieve calls are not billed.

See references/serp-api.md for complete reference including Maps, Trends, Reviews, Lens, Hotels, Flights parameters.


Web Scraper API

Pre-built scrapers for structured data extraction from 100+ platforms. No parsing logic needed.

Sync Endpoint: POST https://api.brightdata.com/datasets/v3/scrape Async Endpoint: POST https://api.brightdata.com/datasets/v3/trigger

# Sync (up to 20 URLs, returns immediately)
response = requests.post(
    "https://api.brightdata.com/datasets/v3/scrape",
    params={"dataset_id": "YOUR_DATASET_ID", "format": "json"},
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={"input": [{"url": "https://www.amazon.com/dp/B09X7M8TBQ"}]}
)

if response.status_code == 200:
    data = response.json()  # Results ready
elif response.status_code == 202:
    snapshot_id = response.json()["snapshot_id"]  # Poll for completion

Parameters

| Parameter | Type | Description | |-----------|------|-------------| | dataset_id | string | Scraper identifier from the Scraper Library (required) | | format | string | json (default), ndjson, jsonl, csv | | custom_output_fields | string | Pipe-separated fields: url\|title\|price | | include_errors | boolean | Include error info in results |

Request Body

{
  "input": [
    { "url": "https://www.amazon.com/dp/B09X7M8TBQ" },
    { "url": "https://www.amazon.com/dp/B0B7CTCPKN" }
  ]
}

Poll for Async Results

import time

# Trigger
snapshot_id = requests.post(
    "https://api.brightdata.com/datasets/v3/trigger",
    params={"dataset_id": DATASET_ID, "format": "json"},
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={"input": [{"url": u} for u in urls]}
).json()["snapshot_id"]

# Poll
while True:
    status = requests.get(
        f"https://api.brightdata.com/datasets/v3/progress/{snapshot_id}",
        headers={"Authorization": f"Bearer {API_KEY}"}
    ).json()["status"]

    if status == "ready": break
    if status == "failed": raise Exception("Job failed")
    time.sleep(10)

# Download
data = requests.get(
    f"https://api.brightdata.com/datasets/v3/snapshot/{snapshot_id}",
    params={"format": "json"},
    headers={"Authorization": f"Bearer {API_KEY}"}
).json()

Progress status values: starting → running → ready | failed Data retention: 30 days. Billing: Per delivered record. Invalid input URLs that fail are still billable.

See references/web-scraper-api.md for complete reference including scraper types, output formats, delivery options, and billing details.


Browser API (Scraping Browser)

Full browser automation via CDP/WebDriver. Handles CAPTCHA, fingerprinting, and anti-bot detection automatically.

Connection:

  • Playwright/Puppeteer: wss://${AUTH}@brd.superproxy.io:9222
  • Selenium: https://${AUTH}@brd.superproxy.io:9515
const { chromium } = require("playwright-core");

const AUTH = process.env.BROWSER_AUTH;
const browser = await chromium.connectOverCDP(`wss://${AUTH}@brd.superproxy.io:9222`);
const page = await browser.newPage();
page.setDefaultNavigationTimeout(120000); // Always set to 2 minutes

await page.goto("https://example.com", { waitUntil: "domcontentloaded" });
const html = await page.content();
await browser.close();
from playwright.async_api import async_playwright

async with async_playwright() as p:
    browser = await p.chromium.connect_over_cdp(f"wss://{AUTH}@brd.superproxy.io:9222")
    page = await browser.new_page()
    page.set_default_navigation_timeout(120000)
    await page.goto("https://example.com", wait_until="domcontentloaded")
    html = await page.content()
    await browser.close()

Custom CDP Functions

| Function | Purpose | |----------|---------| | Captcha.solve | Manually trigger CAPTCHA solving | | Captcha.setAutoSolve | Enable/disable auto CAPTCHA solving | | Proxy.setLocation | Set precise geo location (call BEFORE goto) | | Proxy.useSession | Maintain same IP across sessions | | Emulation.setDevice | Apply device profile (iPhone 14, etc.) | | Emulation.getSupportedDevices | List available device profiles | | Unblocker.enableAdBlock | Block ads to save bandwidth | | Unblocker.disableAdBlock | Re-enable ads | | Input.type | Fast text input for bulk form filling | | Browser.addCertificate | Install client SSL cert for session | | Page.inspect | Get DevTools debug URL for live session |

// CDP session pattern for custom functions
const client = await page.target().createCDPSession();

// CAPTCHA solve with timeout
const result = await client.send("Captcha.solve", { timeout: 30000 });

// Precise geo location (must be before goto)
await client.send("Proxy.setLocation", {
  latitude: 37.7749,
  longitude: -122.4194,
  distance: 10,
  strict: true
});

// Block unnecessary resources
await client.send("Network.setBlockedURLs", { urls: ["*google-analytics*", "*.ads.*"] });

// Device emulation
await client.send("Emulation.setDevice", { deviceName: "iPhone 14" });

Session Rules

  • One initial navigation per session — new URL = new session
  • Idle timeout: 5 minutes
  • Max duration: 30 minutes

Geolocation

  • Country-level: append -country-us to credentials username
  • EU-wide: append -country-eu (routes through 29+ European countries)
  • Precise: use Proxy.setLocation CDP command (before navigation)

Error Codes

| Code | Issue | Fix | |------|-------|-----| | 407 | Wrong port | Playwright/Puppeteer → 9222, Selenium → 9515 | | 403 | Bad auth | Check credentials format and zone type | | 503 | Service scaling | Wait 1 minute, reconnect |

Billing: Traffic-based only. Block images/CSS/fonts to reduce costs.

See references/browser-api.md for complete reference including all CDP functions, bandwidth optimization, CAPTCHA patterns, and debugging.


Detailed References

  • references/web-unlocker.md — Web Unlocker: full parameter list, proxy interface, special headers, async flow, features, billing, anti-patterns
  • references/serp-api.md — SERP API: all Google params (Maps, Trends, Reviews, Lens, Hotels, Flights), Bing params, parsed JSON structure, async, billing
  • references/web-scraper-api.md — Web Scraper API: sync vs async, all parameters, polling, scraper types, output formats, billing
  • references/browser-api.md — Browser API: connection strings, session rules, all CDP functions, geo-targeting, bandwidth optimization, CAPTCHA, debugging, error codes

Related Skills

  • brightdata-proxy — For routing requests through Bright Data's raw proxy networks (Datacenter, ISP, Residential, Mobile) with your own HTTP client instead of a managed API. Covers network/IP-pool selection, the brd-customer-... username format, targeting & sticky-session params, SSL CA setup for Residential/Mobile, and integrations for cURL, Python (requests/httpx/aiohttp/Scrapy), Node (fetch/axios), Playwright, Puppeteer, and Selenium. Hand off to it whenever the task is raw proxy access rather than Web Unlocker / SERP / Web Scraper / Browser API. Escalation order when proxies hit consistent blocks: raw proxy → Web Unlocker → Browser API → Web Scraper API.

相关技能

文档办公

Ontology

为结构化的代理内存和可堆肥技能所打入的知识图. 在创建/征服实体(Person, project, Task, Evention, Document)时使用,链接相关对象,强制约束,规划多步动作作为图变,或技能需要共享状态时使用. 触发到"记住","我知道什么","链接X到Y",…

文档办公

Nano Pdf

使用纳米-pdf CLI编辑带有自然语言指令的PDF.

文档办公

Word / DOCX

创建,检查,并编辑有可靠样式的Microsoft Word文档和DOCX文件,编号,跟踪更改,表格,章节,并进行相容性检查. 当 (1) 任务涉及 Word 或 ".docx " 时使用; (2) 文件包括跟踪的更改,评论,字段,表格,模板,或页面布局限制; (3) 文档必须在不…

文档办公

Api Gateway

通过 Maton 网关调用第三方 API, 它为用户已经连接的应用程序注入证书 。 当用户命名一个连接的应用程序和其中的具体行动时使用此技能——读取一个邮箱,查询一个CMM,归档一个问题,更新一个电子表格,通过一个连接的搜索或刮取提供者运行一个查询. 每一次通话都到用户连接的应用…