做什么
在与双子座Live API构建实时,双向流线应用程序时使用此技能. 封面基于WebSocket的音频/视频/文本流,语音活动检测(VAD),本地音频功能,函数调用,会话管理,客户端认证的ephemer sords,活翻译,以及所有Live API配置选项. SDK覆盖-google-genai (Python), @google/genai (JavaScript/TypeScript).
技能库 智客分类:媒体内容 gemini-live-api-dev
在与双子座Live API构建实时,双向流线应用程序时使用此技能. 封面基于WebSocket的音频/视频/文本流,语音活动检测(VAD),本地音频功能,函数调用,会话管理,客户端认证的ephemer sords,活翻译,以及所有Live API配置选项. SDK覆盖-google-genai (Python), @google/genai (JavaScript/TypeScript).
官方网址:skills.sh
先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。
在与双子座Live API构建实时,双向流线应用程序时使用此技能. 封面基于WebSocket的音频/视频/文本流,语音活动检测(VAD),本地音频功能,函数调用,会话管理,客户端认证的ephemer sords,活翻译,以及所有Live API配置选项. SDK覆盖-google-genai (Python), @google/genai (JavaScript/TypeScript).
官方 description 未单独写出 Use when。按规范,代理会在用户任务与这段 description 的关键词匹配时激活本技能。
按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:Gemini Live API Development Skill、Overview、Models、SDKs、Partner Integrations、Audio Formats。
文件分析:这是一份仅含 SKILL.md 的指令型技能,代理激活后整份正文进入上下文。
Use this skill when building real-time, bidirectional streaming applications with the Gemini Live API. Covers WebSocket-based audio/video/text streaming, voice activity detection (VAD), native audio features, function calling, session management, ephemeral tokens for client-side auth, live translation, and all Live API configuration options. SDKs covered - google-genai (Python), @google/genai (JavaScript/TypeScript).
Gemini Live API Development SkillOverviewModelsSDKsPartner IntegrationsAudio FormatsQuick StartAuthenticationConnecting to the Live APISending TextSending AudioSending Video
来源分类:skills.sh agent-skill
namegemini-live-api-devdescription具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗
先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。
复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。
把 Agent Skill「gemini-live-api-dev」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-e2868bb759cc920f-%E5%8F%8C%E5%AD%90%E5%BA%A7%E6%B4%BB%E4%BD%93-Api-Dev.html 请存为 .cursor/skills/gemini-live-api-dev/SKILL.md 或 .claude/skills/gemini-live-api-dev/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。
需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。
npx skills add 'https://github.com/google-gemini/gemini-skills' --list
npx skills add 'https://github.com/google-gemini/gemini-skills' --skill 'gemini-live-api-dev'
CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。
The Live API enables low-latency, real-time voice and video interactions with Gemini over WebSockets. It processes continuous streams of audio, video, or text to deliver immediate, human-like spoken responses.
Key capabilities:
thinkingLevel)[!NOTE] The Live API currently only supports WebSockets. For WebRTC support or simplified integration, use a partner integration.
gemini-3.1-flash-live-preview — Optimized for low-latency, real-time dialogue. Native audio output, thinking (via thinkingLevel). 128k context window. This is the recommended model for all Live API use cases.gemini-3.5-transcribe-live — Real-time streaming speech-to-text with interim hypotheses, finalized transcripts, smart formatting, and Hybrid VAD.gemini-3.5-live-translate-preview — Real-time streaming translation model.[!WARNING] The following Live API models are deprecated and will be shut down. Migrate to
gemini-3.1-flash-live-preview.
gemini-2.5-flash-native-audio-preview-12-2025— Migrate togemini-3.1-flash-live-preview.gemini-live-2.5-flash-preview— Released June 17, 2025. Shutdown: December 9, 2025.gemini-2.0-flash-live-001— Released April 9, 2025. Shutdown: December 9, 2025.
google-genai — pip install google-genai@google/genai — npm install @google/genai[!WARNING] Legacy SDKs
google-generativeai(Python) and@google/generative-ai(JS) are deprecated. Use the new SDKs above.
To streamline real-time audio/video app development, use a third-party integration supporting the Gemini Live API over WebRTC or WebSockets:
audio/pcm;rate=16000[!IMPORTANT] Use
send_realtime_input/sendRealtimeInputfor all real-time user input (audio, video, and text).send_client_content/sendClientContentis only supported for seeding initial context history (requires settinginitial_history_in_client_contentinhistory_config). Do not use it to send new user messages during the conversation.
[!WARNING] Do not use
mediainsendRealtimeInput. Use the specific keys:audiofor audio data,videofor images/video frames, andtextfor text input.
from google import genai
client = genai.Client(api_key="YOUR_API_KEY")
import { GoogleGenAI } from '@google/genai';
const ai = new GoogleGenAI({ apiKey: 'YOUR_API_KEY' });
from google.genai import types
config = types.LiveConnectConfig(
response_modalities=[types.Modality.AUDIO],
system_instruction=types.Content(
parts=[types.Part(text="You are a helpful assistant.")]
)
)
async with client.aio.live.connect(model="gemini-3.1-flash-live-preview", config=config) as session:
pass # Session is active
const session = await ai.live.connect({
model: 'gemini-3.1-flash-live-preview',
config: {
responseModalities: ['audio'],
systemInstruction: { parts: [{ text: 'You are a helpful assistant.' }] }
},
callbacks: {
onopen: () => console.log('Connected'),
onmessage: (response) => console.log('Message:', response),
onerror: (error) => console.error('Error:', error),
onclose: () => console.log('Closed')
}
});
await session.send_realtime_input(text="Hello, how are you?")
session.sendRealtimeInput({ text: 'Hello, how are you?' });
await session.send_realtime_input(
audio=types.Blob(data=chunk, mime_type="audio/pcm;rate=16000")
)
session.sendRealtimeInput({
audio: { data: chunk.toString('base64'), mimeType: 'audio/pcm;rate=16000' }
});
# frame: raw JPEG-encoded bytes
await session.send_realtime_input(
video=types.Blob(data=frame, mime_type="image/jpeg")
)
session.sendRealtimeInput({
video: { data: frame.toString('base64'), mimeType: 'image/jpeg' }
});
[!IMPORTANT] A single server event can contain multiple content parts simultaneously (e.g., audio chunks and transcript). Always process all parts in each event to avoid missing content.
async for response in session.receive():
content = response.server_content
if content:
# Audio — process ALL parts in each event
if content.model_turn:
for part in content.model_turn.parts:
if part.inline_data:
audio_data = part.inline_data.data
# Transcription
if content.input_transcription:
print(f"User: {content.input_transcription.text}")
if content.output_transcription:
print(f"Gemini: {content.output_transcription.text}")
# Interruption
if content.interrupted is True:
pass # Stop playback, clear audio queue
// Inside the onmessage callback
const content = response.serverContent;
if (content?.modelTurn?.parts) {
for (const part of content.modelTurn.parts) {
if (part.inlineData) {
const audioData = part.inlineData.data; // Base64 encoded
}
}
}
if (content?.inputTranscription) console.log('User:', content.inputTranscription.text);
if (content?.outputTranscription) console.log('Gemini:', content.outputTranscription.text);
if (content?.interrupted) { /* Stop playback, clear audio queue */ }
The Live API supports real-time, low-latency streaming translation of speech (audio) across 70+ languages. For full details on options and capabilities, see the Live Translate Guide.
gemini-3.5-live-translate-preview — The recommended translation model for all Live Translate use cases.TranslationConfig)To enable translation, specify a TranslationConfig object inside your live session setup:
translation_config on LiveConnectConfig:
config = types.LiveConnectConfig(
response_modalities=[types.Modality.AUDIO],
translation_config=types.TranslationConfig(
target_language_code="es", # Target language code (e.g. es, fr, pl)
echo_target_language=True,
),
input_audio_transcription=types.AudioTranscriptionConfig(),
output_audio_transcription=types.AudioTranscriptionConfig(),
)
translationConfig inside generationConfig:
{
"setup": {
"model": "models/gemini-3.5-live-translate-preview",
"generationConfig": {
"responseModalities": ["AUDIO"],
"translationConfig": {
"targetLanguageCode": "es",
"echoTargetLanguage": true
}
}
}
}
The Live API supports real-time streaming speech-to-text over WebSockets with low-latency interim hypotheses, finalized transcripts, and Hybrid VAD. For full details, see the Live Transcription Guide and Colab Cookbook.
gemini-3.5-transcribe-livesmart: cleans up filler words, resolves inline self-corrections, and structures formatting.verbatim (default): exact word-for-word transcript.config = types.LiveConnectConfig(
response_modalities=["TEXT"],
input_audio_transcription=types.AudioTranscriptionConfig(),
)
async with client.aio.live.connect(model="gemini-3.5-transcribe-live", config=config) as session:
# Stream audio
await session.send_realtime_input(audio=types.Blob(data=chunk, mime_type="audio/pcm;rate=16000"))
# Hybrid VAD: notify turn end on client-detected silence for zero latency
await session.send_realtime_input(audio_stream_end=True)
const session = await ai.live.connect({
model: 'gemini-3.5-transcribe-live',
config: {
responseModalities: ['text'],
inputAudioTranscription: { mode: 'smart' }
},
callbacks: {
onmessage: (msg) => {
if (msg.serverContent?.interimInputTranscription) {
console.log('Interim:', msg.serverContent.interimInputTranscription.text);
}
if (msg.serverContent?.inputTranscription) {
console.log('Final:', msg.serverContent.inputTranscription.text);
}
}
}
});
session.sendRealtimeInput({ audio: { data: chunkBase64, mimeType: 'audio/pcm;rate=16000' } });
session.sendRealtimeInput({ audioStreamEnd: true }); // Hybrid VAD
{
"setup": {
"model": "models/gemini-3.5-transcribe-live",
"generationConfig": {
"responseModalities": ["TEXT"],
"speechConfig": {
"voiceConfig": {}
}
},
"inputAudioTranscription": {
"mode": "smart"
}
}
}
TEXT or AUDIO per session, not both. Native audio models only support audio.When migrating from gemini-2.5-flash-native-audio-preview-12-2025 to gemini-3.1-flash-live-preview:
gemini-2.5-flash-native-audio-preview-12-2025 to gemini-3.1-flash-live-preview.thinkingLevel (minimal, low, medium, high) instead of thinkingBudget. Default is minimal for lowest latency.send_client_content is only for seeding initial context history (set initial_history_in_client_content in history_config). Use send_realtime_input for text during conversation.TURN_INCLUDES_AUDIO_ACTIVITY_AND_ALL_VIDEO instead of TURN_INCLUDES_ONLY_ACTIVITY. If sending constant video frames, consider sending only during audio activity to reduce costs.send_realtime_input for all real-time user input (audio, video, text). Reserve send_client_content only for seeding initial context historyaudioStreamEnd when the mic is paused to flush cached audioIf the search_docs tool (from the Google MCP server) is available, use it as your only documentation source:
search_docs with your query[!IMPORTANT] When MCP tools are present, never fetch URLs manually. MCP provides up-to-date, indexed documentation that is more accurate and token-efficient than URL fetching.
If no MCP documentation tools are available, fetch from the official docs index:
llms.txt URL: https://ai.google.dev/gemini-api/docs/llms.txt
This index contains links to all documentation pages in .md.txt format. Use web fetch tools to:
llms.txt to discover available documentation pageshttps://ai.google.dev/gemini-api/docs/live-session.md.txt)[!IMPORTANT] Those are not all the documentation pages. Use the
llms.txtindex to discover available documentation pages
The Live API supports 70 languages including: English, Spanish, French, German, Italian, Portuguese, Chinese, Japanese, Korean, Hindi, Arabic, Russian, and many more. Native audio models automatically detect and switch languages.
媒体内容
以纳米·香蕉Pro(Gemini 3 Pro Image)来生成/编辑图像. 用于创建/修改请求,包括编辑。 支持文本到图像+图像到图像; 1K/2K/4K; 使用 -- input-image.
媒体内容
从YouTube视频中获取并读取文字记录. 需要汇总视频时使用,回答有关视频内容的问题,或者从中提取信息.
媒体内容
Access ATXP支付API工具用于网页搜索,AI图像生成,音乐创建,视频生成,X/Twitter搜索,电子邮件,代理账户管理. 当用户需要实时网络搜索时使用,AI生成的媒体(图像,音乐,视频),X/Twitter搜索,发送/接收电子邮件,或创建和资助代理账户. 需要通过“ …
媒体内容
YouTube Data API与管理的OAuth的集成. 搜索视频,管理播放列表,访问频道数据,并与评论互动. 用户想与YouTube互动时使用此技能. 对于其他第三方应用,使用api-gateway技能(https://clawhub.ai/byungkyu/api-gate…