Summary of Trajdeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents, by Chen Gong et al.
TrajDeleter: Enabling Trajectory Forgetting in Offline Reinforcement Learning Agents
by Chen Gong, Kecen Li, Jin Yao, Tianhao Wang
First submitted to arxiv on: 18 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Cryptography and Security (cs.CR)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach to offline reinforcement learning (RL) called Trajdeleter, which enables agents to rapidly eliminate the influence of specific trajectories from both training datasets and trained agents. The key idea is to guide the agent to deteriorate performance when encountering states associated with unlearning trajectories, while maintaining original performance levels for other remaining trajectories. The paper also introduces Trajauditor, a method for evaluating whether Trajdeleter successfully eliminates targeted trajectories. Experiments on six offline RL algorithms and three tasks demonstrate that Trajdeleter requires only 1.5% of the time needed for retraining from scratch and effectively unlearns an average of 94.8% of targeted trajectories. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Offline reinforcement learning (RL) trains agents using pre-collected datasets, but often retains unwanted information from specific trajectories. This paper solves this problem by introducing Trajdeleter, a practical approach to eliminating trajectory influence. Trajdeleter makes the agent perform poorly when it sees states associated with unlearning, while keeping its original performance for other parts. The paper also shows how to evaluate if Trajdeleter worked correctly. It uses six offline RL algorithms and three tasks to test Trajdeleter and finds it takes much less time than retraining from scratch. |
Keywords
» Artificial intelligence » Reinforcement learning
