ZICQ
中 Log in / Sign up
Newsroom Research & Papers #ARC-Bench #JEPA #Latent Space Models #Action Ranking #AI Planning

ARC-Bench: Exposing Action Ranking Flaws in Frozen JEPA World Models

Avatar of Mr.Xu

By Mr.Xu Compiled & Reviewed by Editorial

Published: · 15 views

中文阅读 (Chinese) English Version
Co-reading Select any sentence in the article to highlight, annotate, or copy a quote card. Join discussion ↓

Summary:arXiv introduces ARC-Bench, a novel benchmark framework that audits the action ranking capabilities of frozen JEPA-style world models. The study reveals that these models suffer from severe structural flaws in action ranking, with top-scored candidates often being suboptimal. The research demonstrates that closed-loop replanning masks this defect, as reducing replanning frequency leads to a collapse in success rates. ARC-Bench provides a measurement tool and explanation for the limitations of la


Background and Motivation

Latent space-based world models have gained popularity in planning and control tasks. These models select actions by evaluating the distance between candidate actions and the goal embedding in a frozen latent space, assuming that latent closeness correlates with true cost. However, the validity of this assumption remains largely unverified.

The ARC-Bench Framework

To directly audit this assumption, the research team introduces ARC-Bench, a no-leak, fixed-candidate protocol that measures the correctness of action ranking by frozen JEPA-style objectives. The framework is applied to official JEPA-WM checkpoints across navigation and manipulation control tasks.

Key Findings

  1. Severe Flaws: The study reveals that frozen latent space representations suffer from severe structural flaws in action ranking. On official manipulation audits, the top-scored candidate is almost always suboptimal, and the same inversion appears in maze domains.
  2. Visual Backbone Extension: A controlled visual-backbone extension shows that the defect persists when DINOv2 is replaced by video-pretrained V-JEPA 1 and V-JEPA 2 encoders at ViT-L/ViT-G scale.
  3. Masking Effect of Closed-Loop Replanning: Closed-loop replanning masks the defect. Reducing the planner's replanning frequency leads to a collapse in success rates in both navigation and manipulation domains, and episodes rescued by frequent replanning are enriched for severe first-plan ranking failures in the PointMaze first-plan diagnostic.

Explanation and Implications

ARC-Bench's measurements expose the limitations of frozen latent space representations in action ranking, while closed-loop replanning masks these issues by frequently adjusting plans. This finding has significant implications for AI systems relying on latent-space planners, suggesting the need for more direct auditing and verification of action ranking capabilities.

Recommendations for Developers

  • Direct Auditing: In applications relying on latent-space planners, it is recommended to directly audit action ranking capabilities rather than relying solely on closed-loop replanning success rates.
  • Model Improvement: Consider improving model architecture or training methods to enhance the accuracy of latent space representations in action ranking.
  • Benchmarking: Use benchmark frameworks like ARC-Bench to evaluate and compare the performance of different latent space models in action ranking.

Source: ArXiv AI (cs.AI) (2026-09-09)

— END —

Tags: #ARC-Bench #JEPA #Latent Space Models #Action Ranking #AI Planning

Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.

What's your take on this story?
🚀 Explore & Submit Works

Community Comments

Loading live comments and annotations…