arXiv Proposes Minimal-Edit On-Policy Distillation to Address Visual Perception Bottlenecks in Multimodal Reasoning

Summary:arXiv introduces Minimal-Edit On-Policy Distillation, a novel method addressing the perceptual bottlenecks in vision-language models (VLMs) for multimodal reasoning. This approach corrects only the minimal edits in incorrect student trajectories using reference solutions, avoiding full-response imitation and eliminating the need for additional descriptions during inference. Experiments demonstrate its effectiveness in enhancing perception-oriented performance, particularly in low-level geometric

---

Technical Mechanism Analysis

Vision-Language Models (VLMs) often face perceptual bottlenecks in multimodal reasoning tasks, where visual perception errors lead to reasoning based on incorrect premises. The Minimal-Edit On-Policy Distillation method addresses this issue through the following mechanisms:

1. Minimal-Edit Correction: The teacher model scores incorrect student trajectories based on reference figure descriptions and verified solutions, applying corrective loss only to tokens disfavored by the teacher. This avoids full-response imitation while retaining on-policy supervision.

2. Task-Reward Reinforcement Learning: Correct trajectories receive task-reward reinforcement learning to further enhance model performance.

3. Inference Without Descriptions: The method eliminates the need for additional descriptions during inference, relying solely on the training-stage correction mechanism to match or exceed baseline model performance.

Engineering Trade-offs and Performance

• Strengths:

  • Enhanced Perception: Particularly strong in low-level geometric perception tasks.
  • Cross-Domain Transfer: The method transfers well to external mathematics and perception benchmarks, demonstrating broad applicability.
  • Training Efficiency: No need for additional descriptions during inference reduces computational costs.

• Limitations:

  • Teacher Model Scale Dependency: Performance gains are more pronounced with larger teacher models, while gains are smaller with same-size teachers.

Developer Implementation and Deployment Recommendations

1. Model Selection: Opt for larger teacher models when possible to maximize performance gains.
2. Training Data Preparation: Ensure high-quality reference figure descriptions and verified solutions are available to fully leverage the method's strengths.
3. Inference Optimization: Focus on optimizing other aspects of the model, such as inference speed and memory usage, since additional descriptions are not required during inference.

Conclusion

The Minimal-Edit On-Policy Distillation method offers an effective solution to the perceptual bottlenecks in VLMs, particularly in scenarios requiring strong low-level geometric perception. It enhances model performance while maintaining efficient training and inference, providing a new direction for the development of multimodal AI applications.

---

Source: ArXiv Machine Learning (cs.LG) (2026-10-23)

— END —

Tags: #Multimodal Reasoning Follow #Model Distillation Follow #Vision-Language Models Follow #Reinforcement Learning Follow #Perception Optimization Follow