Summary of Comparing Few to Rank Many: Active Human Preference Learning Using Randomized Frank-wolfe, by Kiran Koshy Thekumparampil et al.
Comparing Few to Rank Many: Active Human Preference Learning using Randomized Frank-Wolfe
by Kiran Koshy Thekumparampil, Gaurush Hiranandani, Kousha Kalantari, Shoham Sabach, Branislav Kveton
First submitted to arxiv on: 27 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Information Theory (cs.IT); Optimization and Control (math.OC); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper tackles the challenge of learning human preferences from limited comparison feedback, a problem ubiquitous in machine learning with applications like reinforcement learning from human feedback. The authors formulate this task as learning a Plackett-Luce model over a universe of N choices from K-way comparison feedback, where typically K N. They propose the D-optimal design for the Plackett-Luce objective and develop a randomized Frank-Wolfe (FW) algorithm to solve the linear maximization sub-problems. The algorithm is evaluated empirically on synthetic and open-source NLP datasets, demonstrating its effectiveness in this task. This work has implications for machine learning applications where human feedback is scarce or expensive. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Imagine trying to figure out what people like best from a few hints about their preferences. That’s the challenge this paper solves! The authors want to learn how people prefer things when they only get limited information about those preferences. They use a special type of model and an efficient algorithm to do this. This is important because it can help machines learn from humans in situations where we only have a little feedback. The researchers tested their approach on some datasets and found that it works well. |
Keywords
» Artificial intelligence » Machine learning » Nlp » Reinforcement learning from human feedback
