Summary of Deep Generative Models For Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directions, by Jiayu Chen et al.
Deep Generative Models for Offline Policy Learning: Tutorial, Survey, and Perspectives on Future Directions
by Jiayu Chen, Bhargav Ganguly, Yang Xu, Yongsheng Mei, Tian Lan, Vaneet Aggarwal
First submitted to arxiv on: 21 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Deep generative models (DGMs) have revolutionized various domains, such as text, image, and video generation. Similarly, data-driven decision-making and robotic control require learning a generator function from offline data to serve as the strategy or policy. Offline policy learning has great potential, with numerous studies exploring this direction. However, the field lacks a comprehensive review, leading to independent developments in different branches. This paper provides the first systematic review on DGM applications for offline policy learning, covering five mainstream models: Variational Auto-Encoders, Generative Adversarial Networks, Normalizing Flows, Transformers, and Diffusion Models, as well as their applications in offline reinforcement learning (offline RL) and imitation learning (IL). Each type of DGM-based offline policy learning is distilled into its fundamental scheme, categorized by the usage of the DGM, and sorted out by the development process of algorithms. Future research directions are discussed, offering a hands-on reference for the research progress in DGMs for offline policy learning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Deep generative models can create new text, images, and videos using data from the past. They’re also useful for making decisions based on data or controlling robots. This paper takes a step back to look at all the different ways these models are being used to make decisions without needing more information. It looks at five main types of models: Variational Auto-Encoders, Generative Adversarial Networks, Normalizing Flows, Transformers, and Diffusion Models. Each type is explained in simple terms, along with its use in making decisions or imitating human actions. This paper aims to help people understand the current state of this research area. |
Keywords
* Artificial intelligence * Diffusion * Reinforcement learning
