Summary of Performative Reinforcement Learning in Gradually Shifting Environments, by Ben Rank et al.
Performative Reinforcement Learning in Gradually Shifting Environments
by Ben Rank, Stelios Triantafyllou, Debmalya Mandal, Goran Radanovic
First submitted to arxiv on: 15 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A new framework is proposed to model the impact of deployed Reinforcement Learning (RL) agents on their environment’s dynamics. This generalizes Performative RL (PRL), allowing for scenarios where the environment adjusts to a policy over time. Two algorithms from performative prediction are adapted, and a novel algorithm called Mixed Delayed Repeated Retraining (MDRR) is introduced, which combines samples from multiple deployments in its training. Conditions for convergence are provided, and three metrics are used to compare the algorithms: number of retrainings, approximation guarantee, and number of samples per deployment. Experimental results using a simulation-based testbed show that MDRR converges significantly faster than previous approaches. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A team of researchers created a new way to understand how artificial intelligence (AI) agents affect their environment when they’re used in real-world situations. Right now, AI agents are designed to work in environments that stay the same. But what if the environment changes because of the agent’s actions? The team developed a framework that takes this into account, allowing them to predict how the environment will change over time. They also created two new algorithms and tested them using computer simulations. One algorithm stood out as being especially good at adapting to changing environments. |
Keywords
* Artificial intelligence * Reinforcement learning
