Summary of Reward Centering, by Abhishek Naik et al.
Reward Centering
by Abhishek Naik, Yi Wan, Manan Tomar, Richard S. Sutton
First submitted to arxiv on: 16 May 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents an innovative approach to solving continuing reinforcement learning problems using discounted methods. The key finding is that centering rewards by subtracting out their empirical average leads to substantial performance improvements at commonly used discount factors, with the gain increasing as the discount factor approaches one. Additionally, the study shows that standard methods are significantly impacted when rewards are shifted by a constant, whereas reward-centering methods remain unaffected. To estimate the average reward in off-policy settings, the authors propose a slightly more sophisticated method. The paper’s findings suggest that reward centering is a general idea that can benefit almost every reinforcement-learning algorithm. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Reinforcement learning helps machines learn from rewards and penalties. A new discovery shows that when solving long-term problems, it’s better to subtract out the average reward. This makes the algorithm work much better at commonly used settings. The study also finds that if the rewards are shifted by a constant, standard methods fail, but this new approach stays strong. To make this work in all situations, the authors came up with a simple way to estimate the average reward. |
Keywords
» Artificial intelligence » Reinforcement learning
