Summary of Human Expertise in Algorithmic Prediction, by Rohan Alur et al.
Human Expertise in Algorithmic Prediction
by Rohan Alur, Manish Raghavan, Devavrat Shah
First submitted to arxiv on: 1 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The novel framework introduced in this paper enables the incorporation of human expertise into algorithmic predictions by distinguishing inputs that are indistinguishable to predictive algorithms, or “look the same”. The approach clarifies the problem of human-AI collaboration in prediction tasks, as experts often form judgments based on information not encoded in an algorithm’s training data. The framework provides a natural test for assessing whether experts incorporate side information and a method for selectively incorporating human feedback into algorithmic predictions. It is shown that this method provably improves the performance of any feasible algorithmic predictor and quantifies this improvement precisely. Empirical results demonstrate that although algorithms often outperform humans on average, human judgment can improve algorithmic predictions on specific instances, which can be identified ex-ante. The approach provides a natural way of uncovering heterogeneity and enabling effective human-AI collaboration. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper introduces a new way for computers and people to work together to make better predictions. Right now, algorithms are good at making predictions, but they often don’t take into account things that humans know, like special information that’s not written down anywhere. The researchers came up with a way to fix this by letting experts help the computer make decisions. This helps the algorithm make better predictions on specific cases, even if it doesn’t do as well overall. |
