Summary of Are Self-attentions Effective For Time Series Forecasting?, by Dongbin Kim et al.
Are Self-Attentions Effective for Time Series Forecasting?
by Dongbin Kim, Jinseong Park, Jaewook Lee, Hoki Kim
First submitted to arxiv on: 27 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The abstract introduces a new architecture called Cross-Attention-only Time Series transformer (CATS) that rethinks the traditional Transformer framework for time series forecasting. By eliminating self-attention and leveraging cross-attention mechanisms, CATS improves long-term forecasting accuracy while reducing parameter count and memory usage. The model achieves superior performance with lower mean squared error compared to existing models across various datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper introduces a new architecture for time series forecasting that uses cross-attention instead of self-attention. It’s called Cross-Attention-only Time Series transformer (CATS). This helps it do better at predicting the future, especially in the long term, and uses fewer parameters than other models. The model does this by treating the future as queries and sharing information across different parts of the input data. |
Keywords
» Artificial intelligence » Cross attention » Self attention » Time series » Transformer
