Summary of Exploration and Anti-exploration with Distributional Random Network Distillation, by Kai Yang et al.
Exploration and Anti-Exploration with Distributional Random Network Distillation
by Kai Yang, Jian Tao, Jiafei Lyu, Xiu Li
First submitted to arxiv on: 18 Jan 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper addresses the limitation of Random Network Distillation (RND), a popular exploration algorithm in deep reinforcement learning. RND is effective but often requires more discriminative power in bonus allocation, which leads to “bonus inconsistency.” The authors introduce Distributional RND (DRND), a derivative that enhances exploration by distilling a distribution of random networks and incorporating pseudo counts for improved bonus allocation. This refinement encourages agents to explore more extensively. Experimental results show DRND outperforms original RND in both online exploration scenarios and offline tasks like D4RL. The code is publicly available on GitHub. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper fixes a problem with an algorithm called Random Network Distillation (RND). RND helps machines learn from experiences, but it has a limitation that makes it not always make the best choices. To fix this, the authors created a new algorithm called Distributional RND (DRND). DRND is better because it helps machines explore more and make good choices. The new algorithm works well in different situations and is better than the old one. You can find the code for the new algorithm on GitHub. |
Keywords
* Artificial intelligence * Distillation * Reinforcement learning
