Summary of Exploring the Limits Of Hierarchical World Models in Reinforcement Learning, by Robin Schiewer and Anand Subramoney and Laurenz Wiskott
Exploring the limits of Hierarchical World Models in Reinforcement Learning
by Robin Schiewer, Anand Subramoney, Laurenz Wiskott
First submitted to arxiv on: 1 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Hierarchical Model-Based Reinforcement Learning (HMBRL) framework combines the benefits of better sample efficiency from model-based reinforcement learning (MBRL) with the abstraction capability of hierarchical reinforcement learning (HRL) to efficiently solve complex tasks. The framework consists of a stack of agents that communicate top-down, proposing goals to subordinate agents, and uses hierarchical world models to simulate environment dynamics at various levels of temporal abstraction. A key focus is on exploring static and environment-agnostic temporal abstraction, allowing concurrent training of models and agents throughout the hierarchy. Although the approach did not outperform traditional methods in terms of final episode returns, it successfully facilitated decision-making across two levels of abstraction using compact, low-dimensional abstract actions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary In this paper, researchers develop a new Hierarchical Model-Based Reinforcement Learning (HMBRL) framework to solve complex tasks efficiently. The framework uses hierarchical world models to simulate environment dynamics at different levels and trains a stack of agents that work together to make decisions. One of the key features is the use of static temporal abstraction, which allows for concurrent training of models and agents. While the approach didn’t perform better than traditional methods in terms of final results, it showed promise in making decisions across multiple levels. |
Keywords
» Artificial intelligence » Reinforcement learning
