arXiv Proposes CAFE: Revolutionizing Feature Distillation in Multilingual Speech Recognition

Summary:arXiv's latest research introduces CAFE (Confined-Adapter Feature distillation), a novel approach to address feature distillation challenges in compact multilingual automatic speech recognition (ASR). Traditional methods often improve non-English performance at the expense of data-rich English. CAFE confines the CKA gradient to adapters that English bypasses, addressing the trade-off at the parameter level rather than the loss level. On the Open ASR Leaderboard, CAFE reduces the average WER from

---

Technical Mechanism Analysis

In the field of multilingual automatic speech recognition (ASR), feature distillation is often considered a technique to enhance model performance. However, traditional methods improve non-English performance at the expense of data-rich English. CAFE (Confined-Adapter Feature distillation) addresses this issue through the following mechanisms:

1. CKA Gradient Restriction: CAFE confines the Centered Kernel Alignment (CKA) gradient to adapters, while English data bypasses these adapters. This approach addresses the trade-off at the parameter level rather than the loss level, avoiding the conflict between the CKA gradient and the task gradient in traditional methods.

2. Adapter Design: CAFE's adapter design allows the model to fully leverage the advantages of CKA when processing non-English languages, while avoiding negative impacts on English performance.

3. Training Data and Model Scale: The study uses 564k hours of training data and state-of-the-art model architectures, ensuring that CAFE's findings are relevant to practical applications.

Engineering Trade-offs and Performance

• Performance Improvement: On the Open ASR Leaderboard, CAFE reduces the average WER from 8.55% to 8.02%, significantly improving multilingual ASR performance.
• Stability Enhancement: CAFE makes English performance more stable across checkpoints, avoiding the common performance fluctuation issues in traditional methods.
• Parameter Efficiency: CAFE achieves performance close to a 0.6B parameter teacher model with only 254M parameters, demonstrating its advantage in parameter efficiency.
• Cross-Lingual Generalization: CAFE performs excellently across 48 test sets in 24 languages, showcasing its strong competitiveness in multilingual environments.

Developer Implementation and Deployment Recommendations

1. Adapter Configuration: Developers are advised to adjust adapter configurations based on specific application scenarios to maximize CAFE's performance across different languages.
2. Training Data Selection: Using high-quality, diverse training data, especially for non-English languages, can further enhance CAFE's effectiveness.
3. Model Scale Selection: Although CAFE performs well at 254M parameters, increasing the model scale may bring additional performance benefits if resources permit.

Conclusion

CAFE, through its innovative feature distillation approach, solves key problems in multilingual ASR and demonstrates its great potential in improving performance, stability, and parameter efficiency. This method provides new ideas and directions for the further development of multilingual speech recognition technology.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-10-15)

— END —

Tags: #Multilingual ASR Follow #Feature Distillation Follow #CAFE Follow #arXiv Follow