arXiv Proposes Inverse Reasoning Models for Efficient Post-Training and Verification from Answer-Only Data

Summary:A new study published on arXiv introduces a self-distillation technique using inverse reasoning LLMs to improve post-training efficiency. By training a partner model to generate reasoning traces from answers alone, the approach enhances model performance without requiring costly supervised data like chain-of-thought (CoT) sequences. Experiments on tasks such as Countdown and GSM8K demonstrate significant improvements in reasoning accuracy, particularly for hard prompts, and the inverse model als

---

Technical Mechanism Analysis

The core innovation of this research lies in the introduction of a self-distillation technique using inverse reasoning LLMs. Traditional post-training methods often rely on costly supervised data like chain-of-thought (CoT) sequences, which are not always available. On the other hand, reinforcement learning-based approaches, while avoiding the need for CoT, suffer from high computational costs due to multiple autoregressive steps per optimization step. The proposed method addresses these challenges through the following:

1. Training the Inverse Model: The inverse reasoning LLM is trained to generate the original model's reasoning traces conditioned on the answers. This transforms answer-only data into richer reasoning information.
2. Offline Generation: The generation of reasoning traces is performed offline, avoiding the need for multiple autoregressive steps during optimization and thus reducing computational costs.
3. Self-Distillation Process: The generated reasoning traces are used to distill knowledge into the original model, enhancing its reasoning capabilities.

Engineering Trade-offs and Performance

The method was validated on the Qwen3-0.6B model and tested on the Countdown and GSM8K tasks. The results demonstrate:

• Improved Reasoning: After GRPO training saturation, a single round of offline distillation roughly doubles the pass@1 score on hard prompts.
• Performance Comparison: In the Countdown task, the method outperforms rejection-sampling self-distillation across all hard-tail metrics at every mix ratio. In the GSM8K task, the gains are smaller.
• Verification Application: The inverse model also serves as an accurate verifier of correctness, raising the selection accuracy on Countdown from 0.753 (majority voting) to 0.969.

Developer Deployment Recommendations

1. Applicable Scenarios: This method is particularly suitable for scenarios where only answer data is available, such as continuous post-training and knowledge updates.
2. Computational Resource Optimization: The offline nature of the inverse model generation allows developers to flexibly allocate computational resources, avoiding the latency and costs associated with online generation.
3. Model Verification: The inverse model can be used as an independent verification tool to assess the consistency and correctness of model outputs.

Conclusion

This study demonstrates an innovative self-distillation technique that effectively enhances the reasoning capabilities of post-training LLMs while maintaining low computational costs. The method provides a new pathway for the continuous learning and knowledge update of LLMs, especially in data-efficient application scenarios.

---

Source: ArXiv Machine Learning (cs.LG) (2026-10-15)

— END —

Tags: #Inverse Reasoning Follow #Self-Distillation Follow #Post-Training Follow #LLMs & Foundation Models Follow #Reasoning Follow