Summary of Mapping Out the Space Of Human Feedback For Reinforcement Learning: a Conceptual Framework, by Yannick Metz et al.
Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework
by Yannick Metz, David Lindner, Raphaël Baur, Mennatallah El-Assady
First submitted to arxiv on: 18 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Human-Computer Interaction (cs.HC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a framework for understanding human feedback in reinforcement learning (RL) scenarios. It develops a taxonomy of feedback types based on nine key dimensions, which allows for unifying human-centered, interface-centered, and model-centered aspects. The framework also identifies seven quality metrics of human feedback that influence both the human’s ability to express feedback and the agent’s ability to learn from it. The authors derive requirements and design choices for systems learning from human feedback, relating these to existing work in interactive machine learning. They identify gaps in existing work and future research opportunities. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper helps machines learn by listening to people. It creates a way to categorize different types of feedback that humans give to machines. The framework considers three important aspects: how humans interact with machines, how machines interact with each other, and what machines learn from feedback. The authors also identify things that make human feedback good or bad. They use this understanding to suggest ways to design systems that can learn from people’s feedback. |
Keywords
* Artificial intelligence * Machine learning * Reinforcement learning
