COSMI: Revolutionizing Human-Object Interaction Generation with Compositional Synthesis

Summary:COSMI, a text-to-interaction diffusion transformer, generates complex human-object interaction scenarios by composing single-object interaction clips. It ensures semantic and physical plausibility through contact consistency, mirrored balancing, body transfer, and language and geometric checks. The COSMI dataset includes 222k sequences and 275 hours of multi-object interaction data, nearly thirty times larger than the largest existing dataset. Experiments demonstrate COSMI's superior performance

---

Key Breakthroughs

1. Novel Approach to Multi-Object Interaction Generation: COSMI generates multi-object interaction scenarios by composing single-object interaction clips, addressing the high cost of multi-object data capture in traditional methods.
2. Contact Consistency Checks: Utilizing contact consistency, mirrored balancing, and body transfer techniques to ensure the physical and semantic plausibility of the generated results.
3. Large-Scale Dataset: The COSMI dataset includes 222k sequences and 275 hours of multi-object interaction data, nearly thirty times larger than the largest existing dataset.
4. Strong Generalization Ability: In benchmark tests, COSMI demonstrates excellent performance in handling unseen objects and interaction combinations, showcasing its strong generalization capabilities in complex scenarios.

Technical Highlights

• Compositional Generation Strategy: Ensures the plausibility of generated results through language models and geometric checks.
• Efficient Data Expansion: The COSMI dataset can be extended by adding new datasets or hand-object interaction recordings.
• Diffusion Transformer Architecture: The COSMI method uses weight-shared object slots to generate a variable number of objects and predicts their positions relative to moving body parts.

Industry Impact

The release of COSMI provides new data and methodological support for multimodal AI research, with significant applications in areas such as robotics, virtual reality, and augmented reality. Developers can leverage the COSMI model and dataset to develop smarter and more natural AI systems. Additionally, the compositional generation strategy of COSMI offers new insights for generative models in other fields.

Developer Recommendations

• Data Extension: Developers can try combining the COSMI dataset with existing datasets to further enhance the model's generalization capabilities.
• Model Optimization: Explore combining the COSMI method with other generative models (such as GANs, VAEs) to explore more efficient generation strategies.
• Application Exploration: It is recommended to explore applications in fields such as robotics, virtual reality, and augmented reality, utilizing COSMI's strengths to develop smarter AI systems.

---

Source: Hugging Face Daily Papers (2026-10-02)

— END —

Tags: #COSMI Follow #Multimodal AI Follow #Generative Models Follow #Diffusion Transformer Follow #Multi-Object Interaction Follow