Introspective Diffusion Releases ORL and dlmrt: Significantly Boosts Diffusion Language Model Inference Efficiency

Summary:Introspective Diffusion has released ORL (On-Policy Residual Learning), a novel approach to accelerate diffusion language models (dLLMs) without compromising generation quality, alongside a custom inference engine called dlmrt. ORL employs residual learning and uncertainty reduction techniques to enable models to process more tokens per forward pass, achieving up to 6.9x higher throughput on various model families. The dlmrt engine, optimized with CUDA graphs and NVFP4 support, reaches 2,788 tok

---

Technical Mechanism Analysis

The ORL (On-Policy Residual Learning) approach introduced by Introspective Diffusion consists of two main mechanisms to accelerate diffusion language models (dLLMs):

1. Residual Learning (On-Policy Residual Learning):

   • Unlike traditional diffusion models that predict everything from scratch at each denoising step, ORL retains previous predictions as residual context and progressively refines them in subsequent steps. This reduces redundant computations and improves efficiency.
   • The key challenge is that the distribution of residuals changes during training, creating a mismatch with inference. ORL addresses this by training directly on fresh trajectories generated by the current model, effectively replaying the exact residual states encountered during inference.

2. Uncertainty Reduction (Learning to Reduce Uncertainty Faster):

   • ORL introduces 'Adjacent Entropy Contraction', which encourages each denoising step to reduce uncertainty relative to the previous step.
   • Additionally, ORL gates the loss to avoid unnecessary optimization of already confident predictions, thereby maintaining accuracy while accelerating convergence.

Engineering Trade-offs and Benchmark Results

• Performance Improvements: On the QwenDLM-4B model, ORL combined with the dlmrt inference engine achieves a 6.9x increase in end-to-end throughput, with the HumanEval pass@1 score improving from 82.9% to 89.6%.
• Comparison with Traditional Methods: On a single H100 GPU, QwenDLM-4B + ORL reaches 1,298 tokens/s, significantly outperforming DFlash (746 tokens/s) and EAGLE3 (405 tokens/s).
• Multi-Model Support: ORL is not limited to QwenDLM-4B; it also optimizes LLaDA-2.1-mini and DiffusionGemma 26B MoE, achieving 3x and 1.2x throughput improvements, respectively.
• Inference Engine Optimization: The dlmrt engine, optimized with CUDA graphs and NVFP4 support, achieves 2,788 tokens/s on a single RTX 5090, demonstrating its strong performance in low-latency, high-throughput scenarios.

Developer Implementation and Deployment Recommendations

• Open Source Resources: Introspective Diffusion has open-sourced all code, models, and evaluation pipelines for ORL and dlmrt, allowing developers to access these resources on GitHub.
• Hardware Requirements: ORL and dlmrt have high hardware requirements; it is recommended to deploy them on GPUs with NVIDIA RTX 5090 or higher performance.
• Application Scenarios: ORL and dlmrt are particularly suitable for scenarios requiring efficient text generation and inference, such as real-time dialogue systems, text generation tasks, and intelligent agent decision-making.

Conclusion

The ORL and dlmrt released by Introspective Diffusion significantly boost the inference efficiency of diffusion language models, showcasing their innovation and technical prowess in the AI field. The open-source resources provide powerful tools for AI developers, promoting the practical application of diffusion models.

---

Source: Reddit r/LocalLLaMA (2026-10-16)

— END —

Tags: #Introspective Diffusion Follow #ORL Follow #dlmrt Follow #Diffusion Models Follow #Inference Acceleration Follow