Summary of Goal-space Planning with Subgoal Models, by Chunlok Lo et al.
Goal-Space Planning with Subgoal Models
by Chunlok Lo, Kevin Roice, Parham Mohammad Panahi, Scott Jordan, Adam White, Gabor Mihucz, Farzane Aminmansour, Martha White
First submitted to arxiv on: 6 Jun 2022
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel approach to model-based reinforcement learning is proposed, combining approximate dynamic programming updates with model-free updates, inspired by the Dyna architecture. The paper highlights the limitations of using learned models for background planning, which can lead to inaccurate and invalid state generation. To address this issue, a goal-space planning (GSP) algorithm is developed, constraining background planning to subgoals and learning local, subgoal-conditioned models. This approach is computationally efficient, incorporating temporal abstraction for faster long-horizon planning and avoiding the need to learn transition dynamics entirely. The GSP algorithm is demonstrated to significantly improve the performance of various base learners in different domains. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper explores a new way to teach machines to make decisions. It’s like a game where the machine learns from mistakes, but with a twist. Instead of trying to predict everything that might happen, it focuses on smaller goals and only learns what’s needed for those goals. This makes the process faster and more efficient. The researchers tested their approach and found that it worked well in different situations. |
Keywords
* Artificial intelligence * Reinforcement learning
