Summary of Policy-shaped Prediction: Avoiding Distractions in Model-based Reinforcement Learning, by Miles Hutson et al.
Policy-shaped prediction: avoiding distractions in model-based reinforcement learning
by Miles Hutson, Isaac Kauvar, Nick Haber
First submitted to arxiv on: 8 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Model-based reinforcement learning (MBRL) is a promising approach for optimizing policies efficiently. However, existing methods can be vulnerable to scenarios where predictable yet irrelevant details occupy the model’s capacity, hindering its ability to learn important environment dynamics. This issue affects leading MBRL methods like DreamerV3 and DreamerPro. To address this challenge, we develop a method that focuses the world model’s capacity through synergy between a pretrained segmentation model, task-aware reconstruction loss, and adversarial learning. Our approach outperforms other methods designed to reduce distractor impact and advances robust model-based reinforcement learning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Model-based reinforcement learning (MBRL) is trying to find better ways to learn new skills. Right now, some MBRL methods are stuck because they focus too much on small details that don’t matter for making good decisions. We found that this problem affects many popular MBRL methods. To solve this issue, we created a new way to make the model focus on what’s important by combining three ideas: using a pre-trained model to identify important parts, trying to rebuild the world in a way that makes sense, and learning to be better at rejecting distractions. Our method does better than other ways people have tried to solve this problem and is an improvement for learning new skills. |
Keywords
* Artificial intelligence * Reinforcement learning
