arXiv Introduces LOTUS: A Novel Forecasting-Grounded Pretraining for Time-Series Language Models

Summary:arXiv introduces LOTUS, a novel pretraining principle for time-series language models that emphasizes forecasting future signal evolution to preserve temporal information. LOTUS divides time-series channels into non-overlapping patches, maps them into the LLM embedding space, and predicts the next patch from the hidden states of time-series tokens. This approach, combined with a temporal-spectral patch representation, enhances the model's ability to capture temporal changes and periodic patterns

---

Technical Mechanism Analysis

The core innovation of LOTUS lies in its 'forecasting-grounded' pretraining principle, which operates through the following steps:

1. Time-Series Segmentation and Mapping: The time-series data is divided into non-overlapping patches and mapped into the LLM embedding space. This ensures the model captures local features of the time-series.

2. Temporal-Spectral Feature Fusion: To capture both temporal changes and periodic patterns, LOTUS introduces a temporal-spectral feature representation, combining raw time-series patches with their spectral representations before mapping them into the embedding space.

3. Prediction Task-Driven: The model is trained to predict the next time patch, encouraging the learned representations to retain temporal information. This not only enhances the model's predictive capabilities but also improves its understanding of the dynamic nature of time-series data.

Engineering Trade-offs and Performance

• Strengths:

  • Efficiency: LOTUS demonstrates high efficiency across multiple benchmarks with minimal trainable parameters, significantly reducing computational costs.
  • Performance Improvement: In the TSRBench benchmark, LOTUS achieves an average accuracy of 59.91%, outperforming the strongest baseline by 3.36%.
  • Versatility: The method is compatible with various language model backbones, showcasing its wide applicability.

• Limitations:

  • Data Dependency: LOTUS's performance is highly dependent on the quality and diversity of the time-series data. For noisy or outlier-rich datasets, its performance may be compromised.
  • Computational Resource Requirements: While the number of trainable parameters is low, the temporal-spectral feature fusion and prediction task-driven components still require substantial computational resources, especially when dealing with large datasets.

Developer Deployment and Implementation Recommendations

For developers, LOTUS offers a new approach to time-series pretraining with the following advantages:

• Ease of Integration: LOTUS can be seamlessly integrated with existing LLM architectures, allowing developers to quickly apply it to time-series-related tasks.
• Performance Optimization: By introducing temporal-spectral feature representations, developers can further optimize the model's performance on complex time-series data.
• Resource Efficiency: Due to its efficiency, LOTUS is particularly suitable for resource-constrained environments such as edge computing and mobile devices.

Developers are advised to adjust model parameters based on specific application scenarios and combine data augmentation techniques to enhance the model's robustness to noisy data.

---

Source: ArXiv Machine Learning (cs.LG) (2026-10-15)

— END —

Tags: #Time-Series Follow #Pretraining Follow #LLMs & Foundation Models Follow #Forecasting Follow #arXiv Follow