Summary of Episodic Reinforcement Learning with Expanded State-reward Space, by Dayang Liang et al.
Episodic Reinforcement Learning with Expanded State-reward Space
by Dayang Liang, Yaru Zhang, Yunlong Liu
First submitted to arxiv on: 19 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes an efficient deep reinforcement learning (DRL) framework that leverages episodic control-based model-free methods to improve sample efficiency. By expanding the state-reward space, incorporating historical and current information, and utilizing retrieval states in the input and rewards, the proposed method addresses the limitations of existing EC-based approaches. The framework integrates retrieved MC-returns into immediate rewards, enabling better evaluation of state values using a Temporal Difference loss. Experimental results on Box2d and Mujoco tasks demonstrate the superiority of this approach over recent sibling methods and common baselines. Additionally, the paper verifies its effectiveness in alleviating Q-value overestimation through further experiments. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research creates a new way to make computers learn better by using past experiences. It’s like a memory that helps them remember what happened before, so they can make better decisions now. This is important because it makes the computer learning process more efficient and faster. The new method also helps to solve some problems with previous approaches that made their decisions not as good as they could be. |
Keywords
* Artificial intelligence * Reinforcement learning
