arXiv Releases DoSA: Revolutionizing Sparse Attention for Enhanced Indexing Efficiency and Inference Speed

Summary:arXiv introduces DoSA (Dots Sparse Attention), a novel sparse attention mechanism that enhances efficiency by making computations conditional and preserving exact token selection. The core innovation lies in using a seed set to provide a lower bound on the winning threshold and leveraging nonnegative head projections and joint residual bounds to limit positive contributions, thereby reducing unnecessary computations. Experimental results demonstrate DoSA's superior performance, with GPU implemen

---

Technical Mechanism Analysis

DoSA (Dots Sparse Attention) is a novel optimization for sparse attention mechanisms that addresses the inefficiency of traditional sparse attention computations. Its core innovations include:

1. Conditional Computation: By providing a lower bound on the winning threshold through a seed set, it avoids full scoring of all candidates, thus reducing computational load.
2. Exact Token Selection: Leveraging nonnegative head projections and joint residual bounds to limit positive contributions, while using activation-informed aggregation to bound the magnitude subtracted by negative heads, ensuring the precision of the selection process.
3. Certificate-Guided Scoring Filtering: Using certificates to guide the scoring process, full scoring is directed only towards candidates that remain eligible, further reducing writeback and selection work.

Engineering Trade-offs and Performance

• Performance Improvements: The GPU implementation of DoSA achieves up to 2.74x and 1.69x speedups over SGLang and LiteTopK, respectively.
• Model Applicability: On GLM-5.2 and DeepSeek-V3.2 models, DoSA achieves 1.94x and 1.50x improvements in first-token generation time, demonstrating its strong potential for practical applications.
• Cross-Language and Cross-Task Performance: Experiments cover English, Chinese, and code workloads, indicating that DoSA performs well across different languages and task types.

Developer Deployment and Integration Recommendations

1. Hardware Requirements: Due to the significant acceleration on GPUs, it is recommended to deploy DoSA in environments equipped with high-performance GPUs.
2. Model Integration: Developers can integrate DoSA into existing sparse attention models to improve inference efficiency.
3. Optimization Strategies: It is advisable to combine DoSA with other optimization techniques, such as quantization and pruning, to further enhance model performance.

Conclusion

DoSA brings a new breakthrough to sparse attention mechanisms by enabling conditional computation and precise token selection, significantly improving indexing efficiency and inference speed. Its excellent performance in multiple benchmarks demonstrates its great potential for practical applications, providing a new direction for the further optimization of AI models.

---

Source: ArXiv Machine Learning (cs.LG) (2026-10-14)

— END —

Tags: #Sparse Attention Follow #DoSA Follow #GPU Acceleration Follow #Inference Optimization Follow #arXiv Follow