Summary of On the Sample Efficiency Of Abstractions and Potential-based Reward Shaping in Reinforcement Learning, by Giuseppe Canonaco et al.
On the Sample Efficiency of Abstractions and Potential-Based Reward Shaping in Reinforcement Learning
by Giuseppe Canonaco, Leo Ardon, Alberto Pozanco, Daniel Borrajo
First submitted to arxiv on: 11 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed approach utilizes Potential Based Reward Shaping (PBRS) to address sample inefficiency in Reinforcement Learning (RL). The choice of potential function is crucial for PBRS, and the technique’s effectiveness is often limited by computational constraints. To overcome these limitations, this paper develops an abstraction-based method to automatically generate a suitable potential function. By analyzing the bias introduced by finite horizons in PBRS, novel insights are gained. Performance evaluation on four environments, including a goal-oriented navigation task and three Arcade Learning Environments (ALE) games, demonstrates that the approach can match CNN-based solutions using a simple fully-connected network. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us learn faster in computer games by making better choices. Right now, computers can get good at these games but it takes them a long time to practice. The idea is to help the computer make better decisions from the start. We do this by creating a special “guide” that tells the computer what’s important and what’s not. This guide helps the computer learn faster and make better choices. We tested our idea on four different games and showed that it can be just as good as using more complicated methods like CNNs. |
Keywords
» Artificial intelligence » Cnn » Reinforcement learning
