NVIDIA's 'DreamDojo' Paper Faces Severe Criticism: ICML Spotlight Paper Allegedly Riddled with Bugs
By Mr.Xu Community Post
Published: · 1 views
Summary:NVIDIA's research paper 'DreamDojo,' presented at ICML spotlight, has been criticized for containing critical technical flaws. The paper, which builds on NVIDIA's prior work Cosmos 2.5, proposes a world model for robotics and claims performance improvements through pre-training on 44,000 hours of human data. However, researchers on Reddit discovered multiple critical bugs in the paper's codebase, affecting both pre-training and post-training phases, rendering the experimental results unreliable.
Overview
NVIDIA's research team presented a paper titled 'DreamDojo' at the ICML spotlight, proposing a world model for robotics based on their prior work Cosmos 2.5. The paper claimed performance improvements through pre-training on 44,000 hours of human data. However, the results and codebase of the paper have been called into question by researchers on Reddit, who identified critical flaws.
Key Issues
- Marginal Performance Improvement: The paper reported only a 0.5 dB PSNR improvement over Cosmos 2.5, which is a minimal gain given the extensive resources invested.
- Code Bugs: Researchers discovered multiple critical bugs in both the pre-training and post-training code, rendering the experimental results unreliable.
- Resource Misallocation: The significant data and computational resources used did not yield the expected performance improvements.
Implications for Peer Review
The incident has raised concerns about the ICML peer-review process. The failure to detect such blatant code errors highlights potential shortcomings in the current AI research review mechanisms.
Industry Impact
- Reflection on AI Research Quality: This event underscores the importance of code reproducibility and experimental rigor in AI research, urging the academic community to strengthen code review and data validation.
- Challenge to Review Processes: The peer-review system needs to place greater emphasis on code review and experimental verification to ensure the reliability of research findings.
Recommendations for Developers
- Strengthen Code Review: Conduct thorough code reviews and experimental validations before submitting papers.
- Increase Experimental Transparency: Publicizing more experimental details and code can enhance the credibility of research results.
- Leverage Automated Validation Tools: Use automated tools to detect common code errors and improve research quality.
— END —Source: Reddit r/MachineLearning (2026-10-08)
Tags: #NVIDIA #DreamDojo #ICML #Robotics Model #Code Bugs
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments