Summary of Pre-trained Language Models Improve the Few-shot Prompt Ability Of Decision Transformer, by Yu Yang et al.
Pre-trained Language Models Improve the Few-shot Prompt Ability of Decision Transformer
by Yu Yang, Pan Xu
First submitted to arxiv on: 2 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper introduces the Language model-initialized Prompt Decision Transformer (LPDT), a novel approach that leverages pre-trained language models for meta-offline reinforcement learning (RL) tasks. Building upon the success of Prompt-DT methods, which utilize parts of trajectories from training tasks as prompts to enhance performance on unseen tasks, LPDT aims to address challenges such as data-hungry nature and limited few-shot prompt abilities. The proposed approach initializes with a pre-trained language model, fine-tunes it using Low-rank Adaptation (LoRA), and incorporates prompt regularization to differentiate between tasks based on prompt feature representations. Experimental results demonstrate that initializing with a pre-trained language model significantly enhances the performance of Prompt-DT on unseen tasks compared to baseline methods. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research paper introduces a new way to make computers learn from games they’ve played before, even if those games are very different from the ones they’re trying now. They call this “meta-RL” (metareinforcement learning). The idea is that by using language models – like the ones that can understand human language – and fine-tuning them for specific game-like situations, computers can learn much faster and better than before. This approach is called LPDT (Language model-initialized Prompt Decision Transformer), and it’s meant to help computers make decisions in new situations based on what they learned from old games. |
Keywords
» Artificial intelligence » Few shot » Fine tuning » Language model » Lora » Low rank adaptation » Prompt » Regularization » Reinforcement learning » Transformer
