i14 Releases Laplace 3B: A Linear Attention Model on AWS Trainium2, Breaking the KV Cache Bottleneck
Summary:i14 has released Laplace 3B, a linear attention model trained and served on AWS Trainium2. Utilizing Gated Linear Attention (GLA), Laplace 3B maintains a fixed recurrent state of 29.5MB per sequence, eliminating the growing key-value (KV) cache issue of traditional transformer models. This results in more predictable memory usage and efficient inference costs, particularly beneficial for long-running agents and applications requiring persistent memory.
Core Breakthrough and Architecture Design
Laplace 3B, introduced by i14, is a linear attention model designed to address the growing key-value (KV) cache issue in traditional Transformer models when handling long sequences. Its key architectural features include:
- Fixed Recurrent State: Laplace 3B maintains a fixed recurrent state of 29.5MB per sequence, regardless of context length. This ensures more predictable memory usage and avoids the memory overhead associated with KV caches.
- Gated Linear Attention (GLA): Utilizing GLA, Laplace 3B reduces the computational complexity of attention from quadratic to linear while maintaining the ability to model long-range dependencies effectively.
Technical Mechanism Analysis
The GLA architecture of Laplace 3B achieves efficient computation through the following:
- State Update and Compression: For each token processed, the model updates a fixed-size matrix state that compresses previous context information.
- Query-Key-Value-Gate Projection: The model projects the input token into four components: query (Q), key (K), value (V), and gate (G) using a fused matrix. The gate controls the adaptive update of the state.
- State Readout and Self-Augmentation: During state readout, the model adds a gated copy of the current value to the output to enhance the response to the current token.
Engineering Trade-offs and Performance
On AWS Trainium2, Laplace 3B demonstrates the following performance characteristics:
- Training Speed: The training speed on a single trn2.48xlarge instance is approximately 120,000 tokens/second.
- Inference Throughput: The model generates 11,544 output tokens per second with 1,280 concurrent sequences.
- Memory Efficiency: Due to the fixed recurrent state, Laplace 3B's memory usage does not increase with context length, providing a significant advantage in long-sequence processing tasks.
Developer Deployment and Recommendations
For developers looking to leverage Laplace 3B, the following points are crucial:
- Native PyTorch Support: Laplace 3B is supported by TorchNeuron Native on AWS Trainium2, allowing seamless integration with existing PyTorch code.
- Inference Optimization: It is recommended to use the compiled decode step to improve throughput, while keeping in mind the use of eager mode when exact reproducibility is required.
- Resource Management: The fixed memory usage of Laplace 3B allows for more effective resource allocation, especially in multi-tenant or resource-constrained environments.
Conclusion
Laplace 3B demonstrates the capability of AWS Trainium2 to support non-traditional Transformer architectures. By utilizing the GLA mechanism, Laplace 3B maintains efficient training and inference performance while addressing the memory bottleneck associated with KV caches. This provides a new solution for applications that require handling long sequences or have strict memory usage requirements.
Key Takeaways
- Native PyTorch Support: Trainium2 supports custom architectures, with Laplace 3B utilizing TorchNeuron Native.
- Communication Design Matters: The Muon all_to_all design reduces per-rank communication and boosts training throughput.
- Fixed State Ensures Predictable Inference Memory: Each sequence uses 29.5MB of memory, regardless of context length.
- Profile Before Compiling: Most decode time is due to operator scheduling rather than arithmetic or HBM bandwidth.
- Benchmark Realistic Arrival Patterns: Fixed batches hide the cost of compiling new shapes for changing batch widths.
— END —Source: AWS Machine Learning Blog (2026-10-20)
Tags: #i14 #Laplace 3B #Linear Attention Model #AWS Trainium2 #GLA Architecture
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments