Summary of Addressing the Issue Of Stochastic Environments and Local Decision-making in Multi-objective Reinforcement Learning, by Kewen Ding
Addressing the issue of stochastic environments and local decision-making in multi-objective reinforcement learning
by Kewen Ding
First submitted to arxiv on: 16 Nov 2022
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores multi-objective reinforcement learning (MORL), an extension of conventional Reinforcement Learning (RL) that solves multi-objective problems. The authors propose a vector Q-learning algorithm with a utility function, which captures the user’s preferences for action selection. The study focuses on factors influencing the optimal policy in environments with stochastic state transitions, aiming to maximize the Scalarised Expected Return (SER). Empirical evaluations are conducted on a simple Multi-objective Markov decision process (MOMDP) Space Traders problem, showing that well-designed reward signals improve the original baseline algorithm’s performance. A variant of MORL Q-Learning incorporating global statistics outperforms the baseline method in the original Space Traders problem. However, it remains below 100% effectiveness in finding the desired SER-optimal policy. Option learning is guaranteed to converge to the desired SER-optimal policy but lacks scalability for complex problems. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at a new way of teaching machines to make decisions that consider multiple goals. It’s like having a super-smart personal assistant that can balance different priorities. The researchers tested their approach on a simple game, where the goal is to collect points and avoid losing them. They found that by using a special type of reward system, they could improve the machine’s performance. However, there are still limitations to this approach. The main contribution of this study is understanding how noisy data affects the machine’s ability to make good decisions. |
Keywords
* Artificial intelligence * Reinforcement learning
