Summary of Avoiding Catastrophe in Online Learning by Asking For Help, By Benjamin Plaut et al.
Avoiding Catastrophe in Online Learning by Asking for Help
by Benjamin Plaut, Hanlin Zhu, Stuart Russell
First submitted to arxiv on: 12 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel online learning problem is proposed to minimize the chance of catastrophic mistakes. The approach assumes that each round’s payoff represents the chance of avoiding catastrophe and aims to maximize the product of payoffs while allowing a limited number of queries to a mentor. The authors show that any algorithm either constantly queries the mentor or nearly guarantees catastrophe, but provide an algorithm with approaching 0 regret and query rate as the time horizon grows for learnable policy classes. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary To avoid catastrophic mistakes in online learning, researchers have created a new problem where the goal is to minimize the chance of these mistakes. This is done by looking at each round’s payoff as the chance of avoiding catastrophe, and trying to get the most overall while only asking for help a few times. The authors found that any algorithm either asks for help all the time or almost always causes a big mistake. However, they were able to create an algorithm that gets better and better at this problem over time. |
Keywords
* Artificial intelligence * Online learning
