New Method Improves LLM Time Series Forecasting with Semantic and Temporal Alignment

A new research approach addresses a key challenge in adapting large language models to predict continuous numerical patterns. While LLMs excel at understanding text, translating that capability to forecast time series data—stock prices, weather patterns, energy output—has proven difficult.

The Core Problem#

Most existing techniques simply concatenate textual and numerical information or rely on resource-intensive cross-attention mechanisms. Additionally, standard decomposition techniques often fail to capture the intricate dynamics within time series signals, limiting forecast accuracy.

How NeST Works#

The proposed framework, called NeST (neighborhood-aware semantic alignment and temporal modulation), takes a more sophisticated approach to fusing the two modalities:

• Semantic prototypes: The method generates text-based prototypes that represent neighborhoods within the language model's word embeddings, optimized to match regions of the embedding space.
• Alignment and retrieval: Time series representations are aligned with these prototypes using contrastive learning, then the most relevant text prototypes are selected to guide the forecasting process.
• Adaptive feature modulation: Retrieved prototypes condition a temporal modulation mechanism that scales and shifts time series features, allowing the model to adaptively weight different aspects of the input.

Performance and Generalization#

Experimental results across eight standard benchmarks show consistent improvements, with an average 1.2% reduction in mean squared error for long-term forecasting. The framework also demonstrates strong zero-shot performance—forecasting without task-specific training—achieving a 4.9% average MSE reduction. Beyond controlled datasets, NeST performed well on real-world photovoltaic power forecasting across nine operational systems, improving average R² scores by 3.3%.

Full research paper on arXiv