Summary of Lat-pfn: a Joint Embedding Predictive Architecture For In-context Time-series Forecasting, by Stijn Verdenius et al.
LaT-PFN: A Joint Embedding Predictive Architecture for In-context Time-series Forecasting
by Stijn Verdenius, Andrea Zerio, Roy L.M. Wang
First submitted to arxiv on: 16 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces LatentTimePFN (LaT-PFN), a time series forecasting model that enables zero-shot forecasting by performing in-context learning in latent space. The model combines the Prior-data Fitted Networks (PFN) and Joint Embedding Predictive Architecture (JEPA) frameworks to create a prediction-optimized latent representation of underlying stochastic processes. By leveraging related time series as context and introducing a normalized abstract time axis, LaT-PFN reduces training time and increases versatility for any time granularity and forecast horizon. The model outperforms established baselines in zero-shot predictions and produces informative embeddings of individual time steps and fixed-length summaries. Furthermore, the latent space learns to encode local structures in data, similar to vision transformers. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a new way to predict future events in time series data without needing more training data. They use two existing methods, PFN and JEPA, and combine them with some new ideas. The model can learn from short pieces of data and then make predictions about the rest of the data. It’s better than other models at doing this and it also creates a way to understand what the data is saying. |
Keywords
» Artificial intelligence » Embedding » Latent space » Time series » Zero shot
