Summary of Enhancing Cross-domain Pre-trained Decision Transformers with Adaptive Attention, by Wenhao Zhao et al.
Enhancing Cross-domain Pre-Trained Decision Transformers with Adaptive Attention
by Wenhao Zhao, Qiushui Xu, Linjie Xu, Lei Song, Jinyu Wang, Chunlai Zhou, Jiang Bian
First submitted to arxiv on: 11 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper explores the benefits of pre-training decision transformers (DT) using cross-domain data in offline reinforcement learning (Offline RL). While this approach yields superior performance for short-term planning tasks, its effects on fine-tuning are unclear. Additionally, the authors show that this approach hinders distant information extraction in environments requiring long-term planning, resulting in poorer performance compared to training DT from scratch. To explain these findings, they analyze Markov Matrix’s role in pre-trained attention heads and propose a general method, GPT-DTMA, which integrates Mixture of Attention (MoA) into pre-trained DTs for adaptive learning. Experimental results demonstrate the effectiveness of GPT-DTMA in achieving superior performance in short-term environments and mitigating negative impacts in long-term environments. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how using different kinds of data to train decision transformers (DTs) helps with planning tasks. They found that this approach works well for tasks that require quick decisions, but not as well for tasks that need longer-term planning. The authors think that this is because the way DTs are trained makes them good at focusing on nearby information, but bad at looking further away. To solve this problem, they came up with a new way to train DTs, called GPT-DTMA, which lets them learn to focus on different things depending on what’s needed. |
Keywords
» Artificial intelligence » Attention » Fine tuning » Gpt » Reinforcement learning
