Summary of Off-policy Evaluation From Logged Human Feedback, by Aniruddha Bhargava et al.
Off-Policy Evaluation from Logged Human Feedback
by Aniruddha Bhargava, Lalit Jain, Branislav Kveton, Ge Liu, Subhojyoti Mukherjee
First submitted to arxiv on: 14 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper explores off-policy evaluation using logged human feedback in artificial intelligence and machine learning. The authors investigate whether new model evaluations require fresh human feedback or if existing responses from another model can suffice. They formalize the problem, propose two estimators (model-based and model-free) for policy values, and analyze their unbiasedness. Empirical evaluations demonstrate the effectiveness of these estimators in predicting absolute policy values, ranking them, and optimizing them. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at a way to learn from human feedback without having to collect new information all the time. It’s like trying to get a good grade on an exam by using answers from someone who took the same test before. The researchers define the problem, come up with two ways to do it (one based on models and one that doesn’t need them), and show how well these methods work. |
Keywords
» Artificial intelligence » Machine learning
