Tencent Releases Hy-SimulMT-30B-A3B: Real-Time Bidirectional Chinese-English Speech Translation Model

Summary:Tencent Hunyuan has released Hy-SimulMT-30B-A3B, a real-time bidirectional Chinese-English speech translation model. The model supports real-time speech-to-speech translation, preserves the speaker's vocal characteristics, and ensures accurate message delivery. With a StreamLAAL latency of approximately 3 seconds, it outperforms comparable systems, and its GEMBA score exceeds those of most open-source models and some commercial APIs. Hy-SimulMT aims to enhance communication efficiency in setting

---

Technical Architecture and Core Mechanisms

Hy-SimulMT-30B-A3B, developed by Tencent Hunyuan, is a Transformer-based real-time bidirectional Chinese-English speech translation model. Its key features include:

1. Real-time Speech Processing: The model continuously receives audio input and translates as the speaker talks, outputting both translated text and speech.
2. Voice Cloning: Utilizing real-time voice cloning technology, the model preserves the speaker's vocal characteristics, ensuring the translated speech matches the original in terms of tone and timbre.
3. Low Latency: In the RealSI long-form audio evaluation, Hy-SimulMT achieved a StreamLAAL latency of approximately 3 seconds, significantly lower than comparable systems.
4. High Translation Quality: The GEMBA score indicates that the model's translation quality surpasses most open-source models and some commercial APIs.

Engineering Trade-offs and Performance

• Latency vs. Quality: Hy-SimulMT strikes a balance between latency and translation quality. While its low latency makes it suitable for real-time communication, it may compromise on the precision of complex sentences.
• Computational Resources: Real-time speech translation demands substantial computational resources, especially when processing long audio streams. Tencent claims that Hy-SimulMT runs efficiently with GPU acceleration, but specific hardware requirements are not disclosed.
• Evaluation Environment: The RealSI evaluation used real-world audio data and compared the model's performance with that of professional simultaneous interpreters, lending credibility to the results.

Developer Deployment and Implementation

• Use Cases: Hy-SimulMT is ideal for scenarios like meetings, speeches, and interviews that require real-time translation. Developers can utilize the SimulInfer inference service and SimulTrain training tools for deployment and fine-tuning.
• TTS Service Integration: Speech output and voice cloning require a configured TTS service. Developers can choose from open-source or commercial TTS solutions for integration.
• Hy Translate App: Developers can try Hy-SimulMT in the Hy Translate App to quickly understand its capabilities and application scenarios.

Future Outlook

The release of Hy-SimulMT marks a significant advancement in Tencent's real-time speech translation capabilities. In the future, Tencent may further optimize the model's performance and expand its supported languages and scenarios. As AI technology continues to evolve, the application of real-time speech translation will become more widespread, and Hy-SimulMT is poised to play a crucial role in more fields.

---

Source: Reddit r/LocalLLaMA (2026-10-26)

— END —

Tags: #Tencent Follow #Real-time Translation Follow #Voice Cloning Follow #AI Model Follow #Multimodal Follow