Summary of Think Thrice Before You Act: Progressive Thought Refinement in Large Language Models, by Chengyu Du et al.
Think Thrice Before You Act: Progressive Thought Refinement in Large Language Models
by Chengyu Du, Jinyi Han, Yizhou Ying, Aili Chen, Qianyu He, Haokun Zhao, Sirui Xia, Haoran Guo, Jiaqing Liang, Zulong Chen, Liangyue Li, Yanghua Xiao
First submitted to arxiv on: 17 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A recent breakthrough in large language models (LLMs) reveals that progressive refinement, rather than providing a single answer, leads to more accurate and thoughtful outputs. However, existing methods rely heavily on supervision signals, making it challenging to assess output quality in open-ended scenarios effectively. Furthermore, these methods are typically designed for specific tasks, limiting their generalization to new domains. To address these limitations, the authors propose Progressive Thought Refinement (PTR), a framework that enables LLMs to refine their responses progressively. The PTR framework consists of two phases: first, constructing a high-quality progressive refinement dataset through a collaborative selection strategy, and second, fine-tuning LLMs using a thought-mask structure and adjusting loss weights to encourage refinement. Experimental results demonstrate that PTR significantly enhances LLM performance across ten diverse tasks (avg. from 49.6% to 53.5%) without task-specific fine-tuning. Notably, in more open-ended tasks, LLMs also show substantial improvements in response quality beyond mere accuracy, suggesting that PTR truly teaches LLMs to self-improve over time. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Progressive Thought Refinement (PTR) is a new way for large language models (LLMs) to get better at answering questions. Right now, LLMs are good at providing single answers, but they can make mistakes if the question is open-ended. To fix this, researchers created PTR, which helps LLMs refine their responses over time. This works by having LLMs work together with a “weak” model to build a dataset of high-quality answers, and then adjusting how the model learns from its own mistakes. When tested on 10 different tasks, PTR showed significant improvements in accuracy (from 49.6% to 53.5%) without needing special training for each task. This means that LLMs can learn to get better at answering questions over time. |
Keywords
» Artificial intelligence » Fine tuning » Generalization » Mask
