Hugging Face Releases DuplexJev: Revolutionizing Full-Duplex Voice Agent Decision Efficiency

Summary:Hugging Face introduces DuplexJev, a novel approach that feeds ASR-encoder hidden states into a frozen LLM for single-token supervised speech decisions. This method eliminates the need for decoding, drastically improving decision efficiency in full-duplex voice agents. On an 8-GPU node, DuplexJev can handle 80 decisions about eight utterances in about 0.1 seconds. Additionally, it achieves 90% accuracy in gender and emotion recognition with a cross-attention connector, while maintaining high QA

---

Technical Mechanism Analysis

The core innovation of DuplexJev lies in feeding the hidden states of the ASR encoder into a frozen LLM through a small connector, enabling single-token supervised speech decisions. This approach eliminates the slow autoregressive decoding process, significantly improving decision efficiency. Specifically, DuplexJev employs the following technical mechanisms:

1. Single-token Supervision: By mapping speech questions to the option distribution of the LLM, DuplexJev avoids complex decoding, achieving efficient decision generation.
2. Cross-Attention Connector: In gender and emotion recognition tasks, DuplexJev uses a cross-attention connector to combine speech features with the semantic understanding of the LLM, significantly improving recognition accuracy.
3. Batch Inference Optimization: On an 8-GPU node, DuplexJev can handle 80 decisions in about 0.1 seconds, demonstrating its efficiency in resource-constrained environments.

Engineering Trade-offs and Performance

While DuplexJev improves decision efficiency, it also faces some trade-offs:

• Balance of Accuracy and Efficiency: Although the single-token supervision method improves efficiency, in certain complex scenarios, its accuracy may be slightly lower than traditional autoregressive decoding.
• Impact of the Connector: The cross-attention connector, while enhancing gender and emotion recognition, causes a slight drop in QA accuracy (from 90% to 89%).

In practice, DuplexJev performs well in handling multi-turn dialogues and complex speech tasks, particularly in terms of real-time performance and resource utilization, showcasing its great potential in real-world applications.

Developer Adoption and Deployment Recommendations

For developers, the release of DuplexJev offers the following advantages:

1. Open Model Weights and Training Recipe: Developers can quickly build and customize their full-duplex voice agents based on the model weights and training recipe provided by Hugging Face.
2. Support for Batch Inference Pipeline: DuplexJev's batch inference pipeline supports high-throughput task processing, suitable for applications requiring high throughput.
3. Bilingual Spoken-QA Dataset: The open bilingual dataset provides rich training resources for developers, helping to improve the model's performance in multilingual environments.

It is recommended that developers flexibly adjust the connector strategy according to specific application scenarios to find the best balance between accuracy and efficiency.

---

Source: Hugging Face Daily Papers (2026-10-02)

— END —

Tags: #Hugging Face Follow #DuplexJev Follow #Full-Duplex Voice Agents Follow #LLMs & Foundation Models Follow #Speech Decisions Follow