Summary of Hierarchical Prompt Decision Transformer: Improving Few-shot Policy Generalization with Global and Adaptive Guidance, by Zhe Wang et al.
Hierarchical Prompt Decision Transformer: Improving Few-Shot Policy Generalization with Global and Adaptive Guidance
by Zhe Wang, Haozhu Wang, Yanjun Qi
First submitted to arxiv on: 1 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Decision transformers are a type of reinforcement learning algorithm that conditionally generates sequences. Recent advancements in this area have focused on using prompts to facilitate few-shot policy generalization. However, current methods rely on static prompt segments, limiting their ability to provide context-specific guidance. To address this, we introduce a hierarchical prompting approach enabled by retrieval augmentation. Our method learns two layers of soft tokens: global tokens that encapsulate task-level information and adaptive tokens that deliver focused, timestep-specific instructions. These adaptive tokens are dynamically retrieved from a curated set of demonstration segments, ensuring context-aware guidance. We evaluate our approach across seven benchmark tasks in the MuJoCo and MetaWorld environments and demonstrate consistent outperformance of all baseline methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Reinforcement learning is a way to train machines to make decisions. Recently, people have been working on ways to teach these machines new skills quickly. One idea is to use prompts to help them learn. But current approaches are limited because they don’t take into account the specific situation or context. To solve this problem, we developed a new approach that uses two types of prompts: one that provides general information and another that gives focused instructions. These prompts are chosen based on what the machine has seen before, allowing it to learn in different situations. We tested our approach on several tasks and found that it performed better than other methods. |
Keywords
» Artificial intelligence » Few shot » Generalization » Prompt » Prompting » Reinforcement learning
