Hugging Face Releases FRAC: A Novel Fractional Dynamics-Based Architecture for Long Sequence Modeling
Summary:Hugging Face has introduced FRAC, a novel selective State Space Model (SSM) architecture based on fractional dynamics, which replaces the traditional exponential forgetting mechanism with power-law long memory to enhance long-sequence modeling capabilities. FRAC approximates the heavy-tailed target kernel with a finite-state sum of exponential modes, enabling efficient parallel training and prefill while retaining bounded-state autoregressive decoding. Experiments demonstrate that FRAC consisten
A New Architecture for Long Sequence Modeling: The Release of FRAC
Hugging Face has recently introduced FRAC, a novel selective State Space Model (SSM) architecture based on fractional dynamics, aiming to address the limitations of traditional SSMs in long-sequence modeling.
Key Technical Features
- Fractional Dynamics Replacing Exponential Forgetting: FRAC employs a power-law long memory mechanism, replacing the traditional exponential forgetting mechanism to retain information over a broader temporal range.
- Efficient Approximation and Implementation: By approximating the heavy-tailed target kernel with a finite-state sum of exponential modes, FRAC achieves efficient parallel training and prefill while retaining bounded-state autoregressive decoding.
- Experimental Validation: In experiments with a 1.3B-parameter language model, FRAC demonstrated superior performance in long-context tasks while remaining competitive in short-context scenarios.
Industry Impact
- A New Direction for Long Sequence Modeling: The release of FRAC provides a new approach to long-sequence modeling, particularly in tasks that require handling long-range dependencies, such as text generation, speech recognition, and biological sequence analysis.
- Advancing AI Technology: This architecture showcases the potential of fractional dynamics in AI models, paving the way for future research.
- Developer Tool Support: Hugging Face plans to provide pre-trained models and open-source implementations of FRAC in subsequent releases, offering developers more powerful tools.
Recommendations for Developers
- Explore FRAC's Applications: Developers can experiment with FRAC in tasks that require long-sequence modeling, such as natural language processing and bioinformatics.
- Engage with the Open-Source Community: Hugging Face encourages developers to participate in the open-source community around FRAC to collectively advance the technology and its applications.
Conclusion
The release of FRAC marks a significant breakthrough in fractional dynamics for long-sequence modeling, opening new possibilities for the further development of AI models.
— END —Source: Hugging Face Daily Papers (2026-09-28)
Tags: #Hugging Face #FRAC #Long Sequence Modeling #Fractional Dynamics #Open Source Model
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments