ZICQ
中 Log in / Sign up
Newsroom LLMs & Foundation Models #SparsePR #Sparse Attention #Video Generation #World Models #Transformer

SparsePR: Training-Free Sparse Attention via Response-Coupled Partitioning and Probe-Fitted Residual Reconstruction

Avatar of Mr.Xu

By Mr.Xu Compiled & Reviewed by Editorial

Published: · 19 views

中文阅读 (Chinese) English Version
Co-reading Select any sentence in the article to highlight, annotate, or copy a quote card. Join discussion ↓

Summary:SparsePR introduces a novel training-free sparse attention approach that combines response-coupled partitioning with probe-fitted residual reconstruction to enhance efficiency in video generation and world models. By forming paired K/V groups from sampled query-key responses and calibrating an affine correction within the sparse output using a small set of exact query rows, SparsePR achieves 1.48x to 2.61x end-to-end speedups while preserving generation quality. Experiments across four heterogen


Background and Challenges

In video generation and world models, traditional block-sparse attention methods can accelerate Transformer models but often suffer from degraded attention quality. Specifically, queries sharing a block route may have poorly overlapping supports, and the retained attention mass alone cannot fully determine the post-softmax error from skipped interactions.

The SparsePR Approach

SparsePR addresses these issues through the following innovations:

  1. Response-Coupled Partitioning: By forming paired K/V groups from sampled query-key responses and using the centroids of these groups to generate query-response coordinates, SparsePR enables shared routing.
  2. Probe-Fitted Residual Reconstruction: A small set of exact query rows calibrates an affine correction within the sparse output, ensuring high precision within the output subspace observed in the probe residuals.

Experimental Results

SparsePR excels in the following aspects:

  • Reduced Attention-Reconstruction Error: Across four different video generation and world models, SparsePR consistently lowers attention-reconstruction error.
  • Speedup: While maintaining generation quality, SparsePR achieves 1.48x to 2.61x speedups.
  • Realized Density: At 22.0%-26.0% realized executed-pair density, SparsePR preserves high-quality generation.

Industry Impact and Developer Recommendations

SparsePR offers a new technical path for sparse attention mechanisms in video generation and world models, particularly for applications demanding high computational efficiency and generation quality. Developers can integrate SparsePR into existing Transformer models to enhance inference speed and efficiency. Additionally, the modular design of SparsePR makes it easy to combine with other optimization techniques for further performance improvements.


Source: Hugging Face Daily Papers (2026-08-19)

— END —

Tags: #SparsePR #Sparse Attention #Video Generation #World Models #Transformer

Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.

What's your take on this story?
🚀 Explore & Submit Works

Community Comments

Loading live comments and annotations…