Summary of Beyond Worst-case Attacks: Robust Rl with Adaptive Defense Via Non-dominated Policies, by Xiangyu Liu et al.
Beyond Worst-case Attacks: Robust RL with Adaptive Defense via Non-dominated Policies
by Xiangyu Liu, Chenghao Deng, Yanchao Sun, Yongyuan Liang, Furong Huang
First submitted to arxiv on: 20 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates policy robustness in reinforcement learning (RL) under state-adversarial attacks, going beyond worst-case scenarios. It formalizes the task at test time as a regret minimization problem and shows that achieving sublinear regret is hard when using a general continuous policy class. To address this challenge, it proposes an algorithm to iteratively discover non-dominated policies, forming a near-optimal and minimal policy class that ensures both robustness and test-time efficiency. Empirical results on the Mujoco benchmark demonstrate the superiority of this approach in terms of natural and robust performance, as well as adaptability to various attack scenarios. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at how to make computer programs that learn from experience (reinforcement learning) work better when they’re attacked by bad guys. Right now, most ways to do this are good at defending against strong attacks, but they often don’t work so well if the attack is weak or there’s no attack at all. The researchers try a new approach that’s different from what others have done. They make sure the computer program is working with a small group of options instead of a big one, and then use an “adversarial bandit” to help it learn how to adapt quickly. This makes the program more robust (able to withstand attacks) and efficient (works well in different situations). The results are better than what others have done on the Mujoco test. |
Keywords
* Artificial intelligence * Reinforcement learning
