Summary of Enhancing Decision Transformer with Diffusion-based Trajectory Branch Generation, by Zhihong Liu et al.
Enhancing Decision Transformer with Diffusion-Based Trajectory Branch Generation
by Zhihong Liu, Long Qian, Zeyang Liu, Lipeng Wan, Xingyu Chen, Xuguang Lan
First submitted to arxiv on: 18 Nov 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A Decision Transformer (DT) is a machine learning model that can learn effective policies from offline datasets by converting the offline reinforcement learning into a supervised sequence modeling task. The DT uses auto-regressive conditioning on the return-to-go (RTG) to generate trajectory elements. However, this approach tends to learn sub-optimal policies that converge within the dataset. To address this issue, the authors introduce Diffusion-Based Trajectory Branch Generation (BG), which expands the trajectories in the dataset with branches generated by a diffusion model. This expansion enables DT to learn policies that move to better trajectories and prevents it from converging to sub-optimal ones. The authors demonstrate the effectiveness of their approach on the D4RL benchmark, outperforming state-of-the-art sequence modeling methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A Decision Transformer is a way for computers to learn how to make good decisions from old data. This helps them make better choices without having to try every option. But sometimes this process gets stuck and doesn’t lead to the best results. To fix this, researchers created a new technique called Diffusion-Based Trajectory Branch Generation. This allows the computer to add more paths or options to the data it’s learning from, making it easier for the Decision Transformer to make better choices. By doing so, they were able to improve the performance of their model and outdo other methods in tests. |
Keywords
» Artificial intelligence » Diffusion » Diffusion model » Machine learning » Reinforcement learning » Supervised » Transformer
