Summary of Stransformer: a Modular Approach For Extracting Inter-sequential and Temporal Information For Time-series Forecasting, by Jiaheng Yin et al.
sTransformer: A Modular Approach for Extracting Inter-Sequential and Temporal Information for Time-Series Forecasting
by Jiaheng Yin, Zhengxin Shi, Jianshen Zhang, Xiaomin Lin, Yulin Huang, Yongzhi Qi, Wei Qi
First submitted to arxiv on: 19 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary In this study, researchers investigate the effectiveness of Transformer-based models in long-term time-series forecasting (LTSF) tasks. Recent studies have raised questions about the superiority of these models over simple linear layers, which surprisingly outperform them. The authors categorize existing Transformer-based models into two types: those modifying the model structure and those preprocessing input data. They propose a new architecture, sTransformer, which combines Sequence and Temporal Convolutional Networks (STCN) to capture both sequential and temporal information. Additionally, they introduce a Sequence-guided Mask Attention mechanism to incorporate global feature information. The authors compare their approach with linear models and existing forecasting models on LTSF tasks, achieving state-of-the-art results. They also demonstrate strong performance on other time-series tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study explores the use of Transformer-based models for predicting future events in long-term time series data. Researchers have questioned whether these advanced models really outperform simpler methods like linear layers. The authors group existing Transformer-based models into two types: those that change how they work and those that prepare the input data differently. They suggest a new approach called sTransformer, which combines different ideas to better capture patterns in the data. The authors test their method against others on long-term forecasting tasks and show it can do better than previous methods. |
Keywords
» Artificial intelligence » Attention » Mask » Time series » Transformer
