Hugging Face Releases Dex-One2Many: Dexterous Manipulation Framework from Single Human Demonstration
Summary:Hugging Face has released Dex-One2Many, a novel real-to-sim-to-real framework designed to address the challenges of generalization and exploration in learning dexterous manipulation from a single human demonstration video. By abstracting the video into sequential scene graphs, Dex-One2Many guides reinforcement learning (RL) for efficient exploration while preserving broad generalization capabilities. Trained entirely in simulation, the framework transfers zero-shot to a real multi-fingered hand
Technical Mechanism Analysis
The core innovation of Dex-One2Many lies in its ability to abstract a single human demonstration video into sequential scene graphs. These graphs serve as generative constraints that guide reinforcement learning (RL) for efficient exploration while maintaining broad generalization capabilities across different object poses and grasp types. Specifically, the scene graphs function in the following ways:
- Generative Constraints: The graphs are used to sample diverse reset states that go beyond the object poses and grasps shown in the video, thus expanding the exploration space.
- Dense Rewards: Each stage provides dense rewards that guide the agent to stay efficient and goal-oriented during exploration.
Engineering Trade-offs and Performance
After training in a simulated environment, Dex-One2Many can transfer zero-shot to a real multi-fingered hand. Its performance across five tool-use and manipulation tasks is as follows:
- Seen Configurations: Dex-One2Many outperforms baseline methods by 6.5% in seen configurations.
- Unseen Scenarios: In unseen scenarios, its generalization capabilities significantly improve, outperforming baseline methods by 71%.
This remarkable generalization ability indicates that Dex-One2Many is highly adaptable and robust when dealing with complex, multi-stage dexterous manipulation tasks.
Developer Implementation and Deployment Recommendations
For developers, Dex-One2Many offers an efficient learning framework that reduces reliance on costly robot demonstrations. Here are some deployment recommendations:
- Data Collection: Use high-quality single human demonstration videos to ensure the accuracy of scene graph generation.
- Simulated Training: Conduct thorough training in a simulated environment to leverage Dex-One2Many's zero-shot transfer capabilities.
- Hardware Adaptation: When deploying on real robots, pay attention to the physical limitations and sensor accuracy of the manipulator to ensure transfer effectiveness.
The release of Dex-One2Many provides a new technical path for the field of dexterous manipulation learning, especially in scenarios where resources are limited.
— END —Source: Hugging Face Daily Papers (2026-10-08)
Tags: #Hugging Face #Dex-One2Many #Reinforcement Learning #Dexterous Manipulation #Sim-to-Real Transfer
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments