Mantis: New Lightweight AI Model Tackles Time Series Classification

Foundation models have transformed machine learning across many fields, but their use in time series classification—the task of categorizing patterns in sequential data—remains surprisingly underdeveloped. Most research has concentrated on forecasting rather than classification. A new paper presents Mantis, a lightweight transformer-based model designed to fill this gap.

The Architecture & Training#

Mantis is built on transformer architecture and trained using only synthetic data through self-supervised contrastive learning, a technique that helps the model learn useful patterns without explicit labels. The researchers identified that how the model breaks down input data into tokens—chunks of information the transformer can process—is crucial to performance. To this end, they developed a novel token generator unit that improves how temporal sequences are converted into a format the model can effectively learn from.

Bridging the Performance Gap#

Rather than relying solely on the base model, the team introduced an enhanced evaluation strategy that combines several techniques: drawing on representations from intermediate layers (not just the final output), assembling predictions from multiple model variants, and blending embeddings across different models. This ensemble-like approach narrows the gap between Mantis and highly specialized, task-specific models.

Results & Impact#

Testing across four collections of diverse datasets representing different real-world applications, Mantis outperformed existing foundation models for time series work, establishing a new performance benchmark in the field. The lightweight design makes it practical for deployment while maintaining competitive accuracy with heavier alternatives.