Hugging Face Releases WOVEN: Revolutionizing Visual World Modeling for Multimodal LLMs
Summary:Hugging Face introduces WOVEN, a dataset and benchmark for training multimodal large language models (MLLMs) in visual transition reasoning. WOVEN organizes transition supervision by scene, action, and reasoning type, addressing systematic deficits in spatial, physical, and temporal reasoning in existing models. Experiments show that training on WOVEN subsets can collectively improve 22 out of 26 external benchmarks by up to 27.3 percentage points and replace 30-50% of a task's training data wit
1. Background and Motivation
Multimodal large language models (MLLMs) struggle with spatial, physical, and temporal reasoning tasks. These issues often stem from a lack of visual transition reasoning capability, where models struggle to understand and predict changes between visual states. To address this, Hugging Face introduces WOVEN, a dataset and benchmark for training visual transition reasoning.
2. Core Mechanisms of WOVEN
WOVEN organizes data through the following:
- Scene Types: Covers 20 different scenes such as urban, forest, indoor, etc.
- Action Types: Includes 5 basic actions like movement, rotation, scaling, etc.
- Reasoning Types: Involves 8 reasoning operations such as causal reasoning, future state prediction, etc.
WOVEN leverages video-pretrained generative models to generate diverse, realistic scene transition data, totaling 36,076 samples.
3. Experiments and Results
The research team evaluated 38 frontier MLLMs (e.g., GPT-5.4 and Qwen3-VL-235B-A22B) and found that even the strongest models fall significantly below human performance in visual transition reasoning tasks, with this deficit being consistent across different model families.
Training MLLMs on WOVEN showed significant improvements:
- Using only about 2000 data subsets for training, 22 out of 26 external benchmarks improved, with the highest improvement being 27.3 percentage points.
- WOVEN data can replace 30-50% of a task's training data without affecting model performance.
4. Training Recommendations and Future Directions
The research team proposes the following training recommendations:
- Select Reasoning Operations over Scenes or Actions: Prioritize supervision data that teaches specific reasoning operations rather than focusing solely on scene or action diversity.
- Prefer Large Changes in Visual State: To enhance model robustness, prioritize data with larger changes in visual state.
5. Technical Value and Impact
The release of WOVEN provides a systematic approach to visual world modeling training for multimodal large language models. Its dataset and training recommendations not only improve model performance in complex reasoning tasks but also pave the way for future research. The widespread adoption of WOVEN is expected to further advance multimodal AI in fields such as robotics, autonomous driving, and smart homes.
— END —Source: Hugging Face Daily Papers (2026-10-08)
Tags: #Hugging Face #Multimodal Models #Visual Reasoning #WOVEN #MLLMs
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments