Summary of Distributional Off-policy Evaluation with Bellman Residual Minimization, by Sungee Hong et al.
Distributional Off-policy Evaluation with Bellman Residual Minimization
by Sungee Hong, Zhengling Qi, Raymond K. W. Wong
First submitted to arxiv on: 2 Feb 2024
Categories
- Main: Machine Learning (stat.ML)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates off-policy evaluation (OPE) in machine learning, focusing on estimating the return distribution of a target policy using offline data generated by a different policy. The authors critique existing approaches that rely on supremum-extended statistical distances, instead proposing a novel method called Energy Bellman Residual Minimizer (EBRM). The paper provides theoretical analyses, including a finite-sample error bound for the EBRM estimator under realizability assumptions and a multi-step extension to improve non-realizable settings. Unlike prior distributional OPE methods, this approach does not require the completeness assumption. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how we can evaluate different policies in machine learning using data from another policy. They’re trying to figure out what will happen if we use a new policy, even if we don’t have data from that exact policy. The authors are focusing on a special type of distance called expectation-extended statistical distances. They propose a new method, EBRM, which gives us better estimates and doesn’t need as much information as other methods do. |
Keywords
* Artificial intelligence * Machine learning
