Summary of Improving Token-based World Models with Parallel Observation Prediction, by Lior Cohen et al.
Improving Token-Based World Models with Parallel Observation Prediction
by Lior Cohen, Kaixin Wang, Bingyi Kang, Shie Mannor
First submitted to arxiv on: 8 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces a novel approach to improving the efficiency and performance of token-based world models (TBWMs), which were inspired by the success of Transformers in processing sequences of discrete symbols. TBWMs consume agent experience as a language-like sequence of tokens, but this sequential generation process can be a bottleneck during imagination, leading to long training times and poor GPU utilization. To resolve this issue, the authors propose a Parallel Observation Prediction (POP) mechanism that augments a Retentive Network (RetNet) with a novel forward mode designed for reinforcement learning settings. The POP mechanism is incorporated into a new TBWM agent called REM (Retentive Environment Model), which achieves superhuman performance on 12 out of 26 games in the Atari 100K benchmark while training in less than 12 hours, demonstrating a 15.4x faster imagination compared to prior TBWMs. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps computers learn more efficiently by improving how they imagine new situations. Computers usually learn from experience, but imagining new experiences can be slow and inefficient. The authors suggest a way to make this process faster and better, called Parallel Observation Prediction (POP). They use POP to create a new computer program that can play games and do tasks much better than before, all while learning quickly. This is important because it could help computers become even more useful in our daily lives. |
Keywords
* Artificial intelligence * Reinforcement learning * Token
