about
Moment: A Family of Open Time-Series Foundation Models (arxiv.org)
55 points by sarusso on May 17, 2024 | hide | past | pdf | 5 comments on HN

In plain words: A set of large models is trained once on a collection of public time-series data, so it can be reused for many tasks instead of training a fresh model for each. With a little labeled data and light fine-tuning, it still worked well.

Abstract · MOMENT: A Family of Open Time-series Foundation Models

We introduce MOMENT, a family of open-source foundation models for general-purpose time series analysis. Pre-training large models on time series data is challenging due to (1) the absence of a large and cohesive public time series repository, and (2) diverse time series characteristics which make multi-dataset training onerous. Additionally, (3) experimental benchmarks to evaluate these models, especially in scenarios with limited resources, time, and supervision, are still in their nascent stages. To address these challenges, we compile a large and diverse collection of public time series, called the Time series Pile, and systematically tackle time series-specific challenges to unlock large-scale multi-dataset pre-training. Finally, we build on recent work to design a benchmark to evaluate time series foundation models on diverse tasks and datasets in limited supervision settings. Experiments on this benchmark demonstrate the effectiveness of our pre-trained models with minimal data and task-specific fine-tuning. Finally, we present several interesting empirical observations about large pre-trained time series models. Pre-trained models (AutonLab/MOMENT-1-large) and Time Series Pile (AutonLab/Timeseries-PILE) are available on Huggingface.

Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, Artur Dubrawski
arXiv:2402.03885 · cs.LG, cs.AI · submitted Feb 6, 2024 · updated Oct 10, 2024
abstract · pdf · html · Accepted at ICML'24. This is a revision. See changelog in the Appendix

add comment on HN

As a layman, what does this do?
While your eyes can naturally spot a pattern over time in time series data, machines can’t. So as we did for imaging, where we pre-trained models to let machines easily identify objects in pictures, now we (they) are doing the same to let machines “see” patterns over time. Then, how these patterns work, this is another story.
Is there code / demos?
To quote the paper:

The models in this family (1) serve as a building block for diverse time series analysis tasks (e.g., forecasting, classification, anomaly detection, and imputation, etc.), (2) are effective out-of-the-box, i.e., with no (or few) particular task-specific exemplars (enabling e.g., zero-shot forecasting, few-shot classification, etc.), and (3) are tunable using in-distribution and task-specific data to improve performance.

https://en.wikipedia.org/wiki/Time_series

https://en.wikipedia.org/wiki/Foundation_model