Summary of A New View on Planning in Online Reinforcement Learning, by Kevin Roice et al.
A New View on Planning in Online Reinforcement Learning
by Kevin Roice, Parham Mohammad Panahi, Scott M. Jordan, Adam White, Martha White
First submitted to arxiv on: 3 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach to model-based reinforcement learning, combining background planning with learned models and model-free updates. The goal is to overcome the limitations of learned models, which can be inaccurate and generate invalid states when iterated many steps. To achieve this, the authors introduce goal-space planning (GSP), which constrains background planning to a set of subgoals and learns local, subgoal-conditioned models. This approach is computationally efficient, incorporates temporal abstraction for faster long-horizon planning, and avoids learning transition dynamics entirely. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary In simple terms, this paper finds a new way to help computers learn from experience by using background information and combining it with model-free learning. The problem with learned models is that they can be wrong and make unrealistic predictions when used many times. To solve this issue, the researchers developed a planning method called goal-space planning (GSP) that limits the use of background information to specific goals and learns small models for each goal. This approach makes it faster and more efficient for computers to learn from experience. |
Keywords
» Artificial intelligence » Reinforcement learning
