Summary of Towards Off-policy Reinforcement Learning For Ranking Policies with Human Feedback, by Teng Xiao et al.
Towards Off-Policy Reinforcement Learning for Ranking Policies with Human Feedback
by Teng Xiao, Suhang Wang
First submitted to arxiv on: 17 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a new approach to optimizing the ranking metric in recommendation systems. Probabilistic learning to rank (LTR) has been widely used, but it can’t maximize long-term rewards. Reinforcement learning models have been proposed, but they achieve inferior accuracy compared to LTR. The authors propose an off-policy value ranking (VR) algorithm that maximizes user long-term rewards and optimizes the ranking metric offline in a unified Expectation-Maximization (EM) framework. The EM process guides the learned policy to integrate future reward and ranking metric, allowing learning without online interactions. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps improve recommendation systems by finding a better way to rank items. Right now, we use a method called probabilistic learning to rank (LTR), but it’s not good at making long-term decisions. Some other approaches have been tried, but they’re not as accurate. The authors of this paper came up with a new idea that can make both short-term and long-term decisions better by using a special type of algorithm. This could help us learn more from our data without needing to interact with users in real-time. |
Keywords
* Artificial intelligence * Reinforcement learning
