Summary of Robust Thompson Sampling Algorithms Against Reward Poisoning Attacks, by Yinglun Xu et al.
Robust Thompson Sampling Algorithms Against Reward Poisoning Attacks
by Yinglun Xu, Zhiwei Wang, Gagandeep Singh
First submitted to arxiv on: 25 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed work aims to enhance the reliability of Thompson sampling, a widely used algorithm for online sequential decision-making, by making it robust against adversarial reward poisoning. The existing algorithms assume uncorrupted rewards, which may not be true in real-world applications where attacks can manipulate the rewards. To address this issue, the authors develop pseudo-posterior calculations that are less susceptible to manipulation by attackers. The proposed algorithms are designed for both stochastic and contextual linear bandit settings, considering scenarios where the agent is aware or unaware of the attacker’s budget. Theoretical analysis shows that these algorithms guarantee near-optimal regret under any attack strategy. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Thompson sampling is a popular way for computers to make decisions in real-world situations. But what if someone tries to trick the computer by giving it fake information? That’s exactly what can happen with “adversarial reward poisoning.” To fix this problem, researchers developed new algorithms that are more resistant to these attacks. They called these new algorithms “robust Thompson sampling.” The scientists tested their ideas in two common scenarios: when the attacker has a limited budget and when they don’t. Their results show that these robust algorithms can still make nearly optimal decisions even with fake information. |
