Summary of State-separated Sarsa: a Practical Sequential Decision-making Algorithm with Recovering Rewards, by Yuto Tanimoto et al.
State-Separated SARSA: A Practical Sequential Decision-Making Algorithm with Recovering Rewards
by Yuto Tanimoto, Kenji Fukumizu
First submitted to arxiv on: 18 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed State-Separate SARSA (SS-SARSA) algorithm is a reinforcement learning method designed for recovering bandits, where rewards depend on the number of rounds elapsed since the last arm pull. Unlike traditional multi-armed bandit algorithms that assume constant rewards, SS-SARSA treats rounds as states and reduces the number of state combinations required for Q-learning/SARSA, making it more efficient for large-scale problems. The algorithm makes minimal assumptions about the reward structure and offers lower computational complexity. Asymptotic convergence to an optimal policy is proved under mild assumptions. Simulation studies show superior performance across various settings. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a new way to learn from rewards that depend on how long it’s been since you last pulled an arm. This can happen in real-life situations where the reward changes over time, like when you’re trying to figure out which type of candy is most popular at a fair. The new algorithm, called SS-SARSA, makes fewer assumptions about how rewards work and does calculations more quickly than other methods. It’s also proven to be very good at finding the best way to do something in the long run. |
Keywords
* Artificial intelligence * Reinforcement learning
