ZICQ
中 Log in / Sign up
Newsroom LLMs & Foundation Models #Non-Verbal Vocalizations #Preference Optimization #Text-to-Speech #DPO #NV-CER

ArXiv Research: Preference Optimization for Non-Verbal Vocalization Synthesis

Avatar of Mr.Xu

By Mr.Xu Compiled & Reviewed by Editorial

Published: · 2 views

中文阅读 (Chinese) English Version
Co-reading Select any sentence in the article to highlight, annotate, or copy a quote card. Join discussion ↓

Summary:This research focuses on the synthesis of non-verbal vocalizations (NVs) such as laughter, coughs, and sighs. By systematically analyzing preference optimization methods—including preference signals, pair construction, and DPO-based optimization objectives—the study introduces the Non-Verbal Character Error Rate (NV-CER) as an evaluation metric. Experiments on the Emilia-NV and augmented NV-Bench datasets, covering 18 NV types, reveal how different design choices affect NV realization and lexica


Background and Motivation

Non-verbal vocalizations (NVs), such as laughter, coughs, and sighs, are essential components of expressive text-to-speech (TTS) systems. However, the effectiveness of preference optimization in NV generation remains poorly understood. This study aims to systematically explore preference optimization methods for NV synthesis to enhance the expressiveness of TTS systems.

Methods and Experiments

  1. Non-Verbal Character Error Rate (NV-CER): The research team introduced a new evaluation metric, NV-CER, which treats NV tags as distinct output symbols and computes a weighted pinyin-based CER to enable controllable optimization of NV realization.
  2. Datasets and Experiments: Experiments were conducted on the Emilia-NV and augmented NV-Bench datasets, covering 18 NV types. The experiments focused on preference signals, pair construction, and DPO-based optimization objectives.
  3. Evaluation Methods: Objective evaluation, LLM-based evaluation, and human evaluation were combined to validate the impact of different design choices on NV realization and lexical fidelity.

Key Findings

  • Effectiveness of NV-CER: NV-CER effectively captures the quality of NV realization and guides the optimization process.
  • Impact of Design Choices: Different preference signals and pair construction methods significantly affect the quality of NV realization.
  • Advantages of DPO Method: The standard DPO method performs well in handling NV synthesis tasks, providing reliable performance improvements.

Industry Impact and Recommendations

This study provides new ideas and methods for post-training optimization of NV synthesis. Developers can refer to the NV-CER metric and DPO method to enhance the expressiveness and user satisfaction of TTS systems. Additionally, the research emphasizes the importance of multimodal data fusion and cross-domain knowledge integration in NV synthesis, pointing out future research directions.

Technical Highlights

  • Innovative Evaluation Metric: Introduces NV-CER as an evaluation metric for NV synthesis, filling a gap in existing methods.
  • Systematic Optimization Method: Provides a comprehensive optimization framework through systematic analysis of preference signals, pair construction, and DPO optimization objectives.
  • Multi-Dimensional Evaluation: Combines objective, LLM-based, and human evaluation to ensure the reliability and comprehensiveness of the research findings.

Source: ArXiv cs.AI (2026-08-25)

— END —

Tags: #Non-Verbal Vocalizations #Preference Optimization #Text-to-Speech #DPO #NV-CER

Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.

What's your take on this story?
🚀 Explore & Submit Works

Community Comments

Loading live comments and annotations…