Summary of Long-term Off-policy Evaluation and Learning, by Yuta Saito et al.
Long-term Off-Policy Evaluation and Learning
by Yuta Saito, Himan Abdollahpouri, Jesse Anderton, Ben Carterette, Mounia Lalmas
First submitted to arxiv on: 24 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper addresses the challenge of estimating the long-term outcomes of an algorithm using only historical and short-term experiment data. The authors propose a new framework called Long-term Off-Policy Evaluation (LOPE), which is based on reward function decomposition. LOPE relaxes the assumption required by existing approaches, allowing it to effectively leverage short-term rewards to reduce variance. Synthetic experiments demonstrate that LOPE outperforms existing methods, particularly when surrogacy is severely violated and the long-term reward is noisy. Real-world experiments on large-scale A/B test data from a music streaming platform show that LOPE can estimate the long-term outcome of actual algorithms more accurately than feasible existing methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper tries to solve a problem where we want to predict how well an algorithm will perform in the future, but we only have information about what it did in the past. The authors propose a new way to do this called Long-term Off-Policy Evaluation (LOPE). LOPE is better than existing methods because it can use short-term data and doesn’t make unrealistic assumptions. This approach was tested using fake experiments and real-world data from a music streaming service, showing that it’s more accurate than other methods. |
