Summary of Distributionally Robust Off-dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximation, by Zhishuai Liu et al.
Distributionally Robust Off-Dynamics Reinforcement Learning: Provable Efficiency with Linear Function Approximation
by Zhishuai Liu, Pan Xu
First submitted to arxiv on: 23 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper tackles off-dynamics Reinforcement Learning (RL), where a policy trained on one domain is deployed to another distinct domain. To solve this problem, the authors propose online distributionally robust Markov decision processes (DRMDPs) that interact with the source domain while optimizing performance under uncertainty sets of the transition kernel. The study introduces DR-LSVI-UCB, an efficient algorithm for off-dynamics RL with function approximation, and establishes a polynomial suboptimality bound. Experimental results demonstrate the algorithm’s robustness and performance. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Off-dynamics Reinforcement Learning (RL) is when a policy trained on one place is used in another different place. The problem is: how to make this work well? To solve it, scientists use something called online distributionally robust Markov decision processes (DRMDPs). These DRMDPs learn and adapt by interacting with the first place while trying to do well in the second place. This paper introduces a new way of doing this called DR-LSVI-UCB and shows that it works well. |
Keywords
* Artificial intelligence * Reinforcement learning
