Summary of Reinforcement Learning From Bagged Reward, by Yuting Tang and Xin-qiang Cai and Yao-xiang Ding and Qiyu Wu and Guoqing Liu and Masashi Sugiyama
Reinforcement Learning from Bagged Reward
by Yuting Tang, Xin-Qiang Cai, Yao-Xiang Ding, Qiyu Wu, Guoqing Liu, Masashi Sugiyama
First submitted to arxiv on: 6 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a new approach to reinforcement learning (RL) called RL from Bagged Reward (RLBR), where agents receive a single reward based on a sequence or trajectory of actions, rather than an immediate reward for each action. The authors define this problem as a Bagged Reward Markov Decision Process (BRMDP) and demonstrate that it can be addressed by solving a standard MDP with properly redistributed rewards. However, they find that existing methods are insufficient to address the challenges of RLBR, particularly as bag lengths increase. To overcome these limitations, the authors propose a novel reward redistribution method equipped with a bidirectional attention mechanism, which enables accurate interpretation of contextual nuances and temporal dependencies within each bag. This approach is experimentally shown to consistently outperform existing methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper explores a new way for machines to learn from rewards that are not given right away, but instead depend on the sequence of actions taken. This problem is called RL from Bagged Reward (RLBR). The authors show how to solve this problem by redistributing the reward in a special way. They also find that existing methods don’t work well when the sequence of actions gets longer. To fix this, they propose a new method that uses attention to understand the context and timing of each action. This new approach is tested and shown to be better than others. |
Keywords
* Artificial intelligence * Attention * Reinforcement learning
