Summary of Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error, by Haoran Li et al.
Towards Optimal Adversarial Robust Q-learning with Bellman Infinity-error
by Haoran Li, Zicheng Zhang, Wang Luo, Congying Han, Yudong Hu, Tiande Guo, Shichen Liao
First submitted to arxiv on: 3 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed work aims to address the lack of optimal robust policies in deep reinforcement learning (DRL) agents, which is essential for establishing robust policies against attacks or disturbances. The researchers introduce a consistency assumption of policy (CAP), stating that optimal actions remain consistent with minor perturbations, and prove the existence of a deterministic and stationary ORP aligned with the Bellman optimal policy. They also highlight the importance of minimizing L^{}-norm to achieve ORP, contrasting previous approaches targeting L^{1}-norm. To validate their findings, they train a Consistent Adversarial Robust Deep Q-Network (CAR-DQN) and demonstrate its top-tier performance across various benchmarks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research aims to make deep reinforcement learning agents more robust against attacks or disturbances. The scientists found that the best way to do this is by making sure the agent’s actions stay consistent even when there are small changes. They also discovered that using a certain kind of math problem (called L^{}-norm) helps them find the best approach. To put their ideas into practice, they created a new type of deep learning network called CAR-DQN and showed it worked well on many different tasks. |
Keywords
* Artificial intelligence * Deep learning * Reinforcement learning
