Summary of Injecttst: a Transformer Method Of Injecting Global Information Into Independent Channels For Long Time Series Forecasting, by Ce Chi et al.
InjectTST: A Transformer Method of Injecting Global Information into Independent Channels for Long Time Series Forecasting
by Ce Chi, Xing Wang, Kexin Yang, Zhiyan Song, Di Jin, Lin Zhu, Chao Deng, Junlan Feng
First submitted to arxiv on: 5 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A recent paper proposes a novel approach to multivariate time series (MTS) forecasting using Transformers. The authors observe that while channel-independent Transformer-based models are robust, they neglect the valuable information contained in channel dependencies. To address this issue, they develop an injection method called InjectTST, which combines the benefits of both channel-independent and channel-mixing structures. This model retains a channel-independent backbone but injects global information into individual channels selectively using a channel identifier, global mixing module, and self-contextual attention module. Experimental results show that InjectTST outperforms state-of-the-art models with stable improvements. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A new approach to predicting many kinds of data at once is presented in this paper. Researchers have been using Transformers for this task, but they’ve noticed that some important details are being left out. To fix this, the authors develop a way to add global information into each individual part of the data. This helps the model use all the valuable information available. The new approach is tested and shown to be better than other methods. |
Keywords
* Artificial intelligence * Attention * Time series * Transformer
