Summary of Learning Mirror Maps in Policy Mirror Descent, by Carlo Alfano et al.
Learning mirror maps in policy mirror descent
by Carlo Alfano, Sebastian Towers, Silvia Sapora, Chris Lu, Patrick Rebeschini
First submitted to arxiv on: 7 Feb 2024
Categories
- Main: Machine Learning (stat.ML)
- Secondary: Machine Learning (cs.LG); Optimization and Control (math.OC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The Policy Mirror Descent (PMD) framework is a popular reinforcement learning approach that encompasses numerous algorithms. These algorithms are derived by selecting a mirror map and enjoy finite-time convergence guarantees. While PMD has been widely used, the exploration of its full potential is limited due to a focus on a particular mirror map, the negative entropy, which gives rise to the Natural Policy Gradient (NPG) method. This study empirically investigates whether the choice of mirror map significantly influences PMD’s efficacy and identifies more efficient mirror maps that enhance its performance using evolutionary strategies. The results suggest that mirror maps generalize well across various environments, raising questions about how to best match a mirror map to an environment’s structure and characteristics. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary PMD is a way to teach machines to make good decisions without getting stuck in bad habits. It’s like teaching a robot to play a game, but the robot doesn’t know the rules. PMD helps the robot learn by giving it hints about what’s a good move and what’s not. But different games might need different hints, so this study figured out that the type of hint used can make a big difference in how well the robot plays. They even found some new hints that work better than the old ones! This is important because it helps us understand how to make machines learn more effectively. |
Keywords
* Artificial intelligence * Reinforcement learning
