Hugging Face Releases MLExpert: Revolutionizing Autonomous Machine Learning Engineering with Mixture-of-Experts Model

Summary:Hugging Face has released MLExpert, a 35B Mixture-of-Experts agentic model designed to autonomously optimize Machine Learning Engineering (MLE) workflows, advancing AI for AI (AI4AI) and recursive self-improvement (RSI). Leveraging Hierarchical Agentic Supervised Fine-Tuning (SFT) and Windowed Interaction Reinforcement Learning (WIRL), MLExpert significantly improves the average medal rate from 37.9% to 72.7% on MLE-Bench Lite, demonstrating strong performance in algorithm optimization and task

---

Technical Mechanism Analysis

The core technological breakthroughs of MLExpert lie in the combination of its Mixture-of-Experts (MoE) architecture and reinforcement learning methods:

1. Mixture-of-Experts (MoE) Architecture: MLExpert employs a 35B-scale MoE architecture that uses dynamic routing to select expert sub-models for different tasks. This design maintains high parameter capacity while reducing inference costs and enhancing the model's adaptability to complex tasks.

2. Hierarchical Agentic Supervised Fine-Tuning (SFT): By transferring general tool-use and coding capabilities to MLE tasks, MLExpert achieves fine-grained control over complex experimental workflows.

3. Windowed Interaction Reinforcement Learning (WIRL): WIRL treats multi-turn decisions between successive experiment results as an execution window and assigns a shared outcome-based reward. This approach effectively captures useful supervision signals from long interaction trajectories and learns optimization strategies from intermediate outcomes.

4. Asynchronous Collection and Policy Correction: MLExpert supports policy updates before the trajectory ends, enabling more efficient online learning and error correction.

Engineering Trade-offs and Performance

• Performance Improvement: On the MLE-Bench Lite benchmark, MLExpert boosts the average medal rate of the base model from 37.9% to 72.7%, approaching the performance of larger frontier models like Kimi-K3 and GLM-5.3.

• Generalization Ability: MLExpert excels in NatureBench Lite, agent scaffold optimization, and algorithm optimization tasks, indicating strong generalization of its optimization capabilities.

• Training Cost and Efficiency: Although the MoE architecture requires handling dynamic routing of expert sub-models during training, the asynchronous collection and policy correction mechanisms significantly reduce training latency while maintaining high performance.

Developer Implementation and Deployment Recommendations

1. Task Adaptability: MLExpert is suitable for machine learning engineering tasks that involve complex experimental workflows and long interaction trajectories, such as hyperparameter optimization, model architecture search, and automated experiment design.

2. Resource Requirements: Due to its 35B scale, MLExpert has high hardware resource requirements and is recommended to be deployed in an environment equipped with GPU clusters.

3. Fine-Tuning and Customization: Developers can fine-tune MLExpert according to specific application scenarios to further enhance its performance on particular tasks.

4. Integration and Extension: MLExpert can be seamlessly integrated with existing MLE toolchains and supports the extension of its functionality through a plugin mechanism.

Conclusion

MLExpert represents a significant advancement in the field of autonomous machine learning engineering. By combining a Mixture-of-Experts architecture with reinforcement learning methods, it demonstrates strong capabilities in complex task optimization and generalization. Its release provides a new technical path for the development of AI for AI and offers powerful tool support for developers.

---

Source: Hugging Face Daily Papers (2026-10-15)

— END —

Tags: #Hugging Face Follow #Mixture-of-Experts Follow #Machine Learning Engineering Follow #Reinforcement Learning Follow #AI4AI Follow