ZICQ
中 Log in / Sign up
Newsroom LLMs & Foundation Models #Hugging Face #Large Language Models #Inference Optimization #Sparse Attention

Hugging Face Introduces Declarative Attention: Significantly Reducing Inference Costs for Large Language Models

Avatar of Mr.Xu

By Mr.Xu Compiled & Reviewed by Editorial

Published: · 5 views

中文阅读 (Chinese) English Version
Co-reading Select any sentence in the article to highlight, annotate, or copy a quote card. Join discussion ↓

Summary:Hugging Face researchers introduce Declarative Attention (DA), a novel mechanism designed to optimize the inference efficiency of large language models (LLMs) when handling long contexts. DA enables the model to proactively declare its attention focus, partitioning the generation process into three modes: global, focused, and local. This approach significantly reduces the KV cache reads, achieving a 52.0% and 31.1% reduction in attended tokens for Gemma-4-31B and Qwen-3.6-27B models, respectivel


### Declarative Attention: A New Direction for Optimizing Long-Context Reasoning Large language models (LLMs) face a critical challenge when handling long contexts: although they focus on a small fraction of the context, they must scan the entire KV cache to generate responses. This approach leads to high computational costs, especially when dealing with extremely long contexts, such as 1 million token conversations. #### Core Innovation Hugging Face's research team introduces Declarative Attention (DA), a novel mechanism that optimizes inference efficiency through the following: - **Proactive Declaration of Attention Focus**: The model proactively declares its attention focus during the reasoning process, rather than passively scanning the entire context. - **Three Modes of Partitioning**: The reasoning process is partitioned into three modes: global (), focused (), and local (). - **KV Cache Read Optimization**: The inference engine parses DA declarations as tool calls and skips most KV cache reads. #### Experimental Results In zero-shot evaluations across 15 long-context tasks, DA achieved significant reductions in attended tokens for Gemma-4-31B and Qwen-3.6-27B models: - Gemma-4-31B: 52.0% reduction in attended tokens, with a 1.27 percentage point drop in accuracy. - Qwen-3.6-27B: 31.1% reduction in attended tokens, with a 2.75 percentage point drop in accuracy. Notably, the accuracy loss diminishes as the model size increases, demonstrating DA's potential for larger models. #### Technical Highlights - **New Dimension of Sparse Attention**: DA opens up a new dimension of sparse attention, surpassing traditional pre-selection methods for tokens. - **Potential for Training-Based Methods**: Future research can further explore DA's potential through training-based methods to enhance model performance. #### Industry Impact DA's introduction brings new possibilities to the AI field, particularly in long-context processing and inference efficiency optimization. Its application scenarios include: - **Conversational AI Systems**: Improving the efficiency of long-dialogue processing. - **Text Generation Tasks**: Reducing computational costs and increasing generation speed. - **Multimodal Models**: Optimizing cross-modal attention mechanisms. ### Developer Recommendations - **Experiment with DA**: Developers can experiment with integrating DA into existing models to improve inference efficiency. - **Stay Updated on Future Research**: Keep an eye on research related to DA-based training methods to further optimize model performance. - **Evaluate Model Scale Impact**: Test DA on models of different scales to assess its applicability. --- > **Source**: [Hugging Face Daily Papers](https://huggingface.co/papers/2609.02737) (2026-09-02) — END —

Tags: #Hugging Face #Large Language Models #Inference Optimization #Sparse Attention

Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.

What's your take on this story?
🚀 Explore & Submit Works

Community Comments

Loading live comments and annotations…