FD-VAD: Semantic Endpoint Detection for Streaming Full-Duplex Speech
Summary:FD-VAD introduces a novel ASR-free streaming endpointing technique for full-duplex voice interaction. By formulating semantic endpoint detection as a causal audio-language reasoning task, FD-VAD maps bounded causal audio windows directly to Continue/Stop decisions, bypassing the limitations of acoustic voice activity detection and cascaded ASR-based methods. The system combines a frozen speech encoder, a lightweight modality adapter, and a parameter-efficiently adapted language model, achieving
Background and Challenges
In full-duplex voice interaction, identifying semantic endpoints (i.e., distinguishing between hesitations and completed conversational intents) in real-time is crucial for enabling natural dialogue. Traditional acoustic voice activity detection (VAD) methods lack semantic understanding, while cascaded ASR-based endpointing introduces transcription dependence and additional processing stages, impacting efficiency and accuracy.
Key Features of FD-VAD
- ASR-free Design: FD-VAD operates directly on audio signals without relying on speech transcription, reducing computational costs and improving processing speed.
- Causal Audio-Language Reasoning: By mapping audio segments to Continue/Stop decisions, FD-VAD achieves direct semantic understanding.
- Modular Architecture: The system combines a frozen speech encoder, a lightweight modality adapter, and a parameter-efficiently adapted language model, ensuring flexibility and efficiency.
- Streaming Inference Optimization: The last-chunk training objective enhances the real-time performance and accuracy of streaming inference.
- Boundary-Focused Hard Negative Sampling: By applying hard negative sampling around ambiguous turn boundaries, FD-VAD significantly improves decision reliability.
Experimental Results
FD-VAD demonstrates superior performance across multiple benchmarks, particularly achieving an EOT recall of 0.853 on the TurnBench dev set (at FP<=0.10), outperforming existing strong streaming and non-streaming semantic turn classifiers. This indicates that FD-VAD can perform semantic endpointing directly from streaming audio without intermediate ASR or dialogue state tracking.
Industry Impact and Developer Recommendations
The release of FD-VAD provides a more efficient and accurate semantic endpoint detection solution for full-duplex voice interaction systems, especially for applications requiring high real-time performance and accuracy, such as intelligent customer service, virtual assistants, and real-time translation systems. Developers can leverage FD-VAD's architecture and implementation to enhance the dialogue processing capabilities of existing systems and explore its potential in multimodal interaction scenarios.
— END —Source: ArXiv NLP/LLM (cs.CL) (2026-10-01)
Tags: #Speech Recognition #Semantic Understanding #Streaming Processing #Intelligent Agents #Dialogue Systems
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments