Summary of Provably Efficient Action-manipulation Attack Against Continuous Reinforcement Learning, by Zhi Luo et al.
Provably Efficient Action-Manipulation Attack Against Continuous Reinforcement Learning
by Zhi Luo, Xiyuan Yang, Pan Zhou, Di Wang
First submitted to arxiv on: 20 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates vulnerabilities in reinforcement learning (RL) in Cyber-Physical Systems (CPS), particularly in the context of autonomous driving. The authors focus on action-manipulation attacks, where an attacker manipulates the agent’s training and behavior by controlling its interaction trajectories with the environment. Existing work has mainly focused on discrete state-action spaces, but this paper addresses the more challenging continuous case. The authors propose a black-box attack algorithm, LCBT, which uses Monte Carlo tree search to efficiently manipulate actions. They also demonstrate that LCBT can teach an agent to converge to target policies with sublinear attack cost. The proposed methods are tested on three aggressive RL algorithms (DDPG, PPO, and TD3) in continuous settings, showing promising results. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at how attackers might manipulate the behavior of artificial intelligence (AI) agents that learn by interacting with their environment. In a cyber-physical system like an autonomous car, this could have bad consequences. Researchers have studied these kinds of attacks before, but only in situations where choices are simple or limited. Now, they’re exploring what happens when there’s a wider range of possibilities. The authors came up with a new way to do these attacks, which works even if you don’t know the details of how the AI agent is designed. They tested their method on three different AI algorithms and found that it was effective. |
Keywords
* Artificial intelligence * Reinforcement learning
