Summary of Planning Transformer: Long-horizon Offline Reinforcement Learning with Planning Tokens, by Joseph Clinton et al.
Planning Transformer: Long-Horizon Offline Reinforcement Learning with Planning Tokens
by Joseph Clinton, Robert Lieck
First submitted to arxiv on: 14 Sep 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a novel approach to offline reinforcement learning, building upon existing Decision Transformer models. By introducing “Planning Tokens” that contain high-level information about the agent’s future actions, the authors are able to reduce compounding error and improve performance on long-horizon tasks. This architectural modification enables the model to use Planning Tokens as implicit planning guides, leading to a state-of-the-art performance in complex D4RL environments. Additionally, the paper shows that Planning Tokens enhance interpretability through visualizations and attention maps. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper helps us learn better from past experiences without actually playing the game. It’s like having a smart assistant that can predict what we’ll need to do next based on what happened before. The new approach uses something called “Planning Tokens” which give the model long-term goals and help it make better decisions. This makes the model much better at solving complex problems, and also helps us understand why it’s making certain choices. |
Keywords
» Artificial intelligence » Attention » Reinforcement learning » Transformer
