Kandinsky 6.0 Video发布：革新同步音视频生成技术

摘要：Kandinsky团队发布了Kandinsky 6.0 Video，这是一系列用于同步文本生成音视频的基础扩散模型，包括Kandinsky 6.0 Video Lite（3B参数）和Kandinsky 6.0 Video Pro（29B参数）。新模型支持5秒视频片段生成，并提供唇语同步功能，同时内置超分辨率模型将输出分辨率提升至全高清（1920x1080）。通过双流CrossDiT架构，新模型实现了视频和音频流之间的时序和语义对齐，并在语音质量方面表现出色。Kandinsky 6.0 Video Pro在人类评估中明显优于其前身Kandinsky 5.0 Video Pro，并与其他领先音视频生成模型竞争。该模型的开源发布旨在加速多媒体生成领域的研究与应用。

---

技术亮点解析

1. 模型架构与功能

   • **Kandinsky 6.0 Video Lite（3B参数）和Kandinsky 6.0 Video Pro（29B参数）**是两款用于同步文本生成音视频的基础扩散模型，支持5秒视频片段生成，并提供唇语同步功能。
   • 内置超分辨率模型将输出分辨率提升至全高清（1920x1080）。
   • 采用双流CrossDiT架构，通过双向交叉注意力连接预训练的视频流和全新训练的音频流，实现时序和语义对齐。

2. 训练策略

   • 连续预训练策略：首先在大型音频语料库上从头训练音频流，然后联合训练视频和音频流，同时保持单模态保真度。
   • 监督微调：在配对音视频数据上进行监督微调。
   • 强化学习后训练：通过强化学习进一步优化模型性能。
   • 蒸馏：采用蒸馏技术提升模型效率。

3. 性能表现

   • 在人类评估中，Kandinsky 6.0 Video Pro明显优于其前身Kandinsky 5.0 Video Pro，并在语音质量方面与其他领先模型竞争。

行业影响与开发者建议

• 多媒体生成领域的突破：Kandinsky 6.0 Video的发布标志着音视频生成技术的重大进步，为多媒体内容创作提供了更强大的工具。
• 开源推动研究与应用：模型的开源发布将加速研究进展，并促进多媒体生成技术的广泛应用。
• 开发者建议：开发者可以利用该模型进行音视频生成任务，并结合其他AI工具构建更复杂的多媒体应用。建议关注模型的训练细节和优化策略，以充分发挥其性能潜力。

英文内容

Technical Highlights

1. Model Architecture and Features

   • Kandinsky 6.0 Video Lite (3B parameters) and Kandinsky 6.0 Video Pro (29B parameters) are foundation diffusion models for synchronized text-to-audio-video generation, supporting 5-second video clips with lip-sync in T2AV and I2AV modes.
   • A built-in super-resolution model boosts the output resolution to Full-HD (1920x1080).
   • The dual-stream CrossDiT architecture connects a pretrained video stream and a newly trained audio stream through bidirectional cross-attention for temporal and semantic alignment.

2. Training Strategy

   • Continuous Pretraining: The audio stream is first trained from scratch on large-scale audio corpora, followed by joint training with the video stream on paired audio-video data while preserving unimodal fidelity.
   • Supervised Fine-Tuning: The model is fine-tuned on paired audio-video data.
   • Reinforcement Learning Post-Training: Further optimized through reinforcement learning.
   • Distillation: Distillation techniques are employed to enhance model efficiency.

3. Performance

   • In side-by-side human evaluation, Kandinsky 6.0 Video Pro outperforms its predecessor, Kandinsky 5.0 Video Pro, and remains competitive with leading audio-video generation models, particularly in speech quality.

Industry Impact and Recommendations for Developers

• Breakthrough in Multimedia Generation: The release of Kandinsky 6.0 Video represents a significant advancement in audio-video generation technology, providing a powerful tool for multimedia content creation.
• Open-Source for Research and Application: The open-source release will accelerate research progress and facilitate the widespread application of multimedia generation technology.
• Developer Recommendations: Developers can leverage this model for audio-video generation tasks and combine it with other AI tools to build more complex multimedia applications. It is recommended to pay attention to the training details and optimization strategies to fully exploit the model's performance potential.

---

消息来源：Hugging Face Trending Papers (2026-10-04)

—— 完 ——

主题标签： #Kandinsky 关注 #音视频生成 关注 #扩散模型 关注 #多模态AI 关注 #开源模型 关注