Summary of Dots: Learning to Reason Dynamically in Llms Via Optimal Reasoning Trajectories Search, by Murong Yue et al.
DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Search
by Murong Yue, Wenlin Yao, Haitao Mi, Dian Yu, Ziyu Yao, Dong Yu
First submitted to arxiv on: 4 Oct 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed DOTS (Dynamic Optimal Trajectory Search) approach enhances the capability of large language models (LLMs) in reasoning by allowing them to reason dynamically via optimal reasoning trajectory search. This is achieved through three key steps: defining atomic reasoning action modules, searching for the optimal action trajectory for each training question, and using the collected optimal trajectories to train an LLM to plan for the reasoning trajectories of unseen questions. Two learning paradigms are proposed: fine-tuning an external LLM as a planner or directly fine-tuning the task-solving LLM with an internalized capability for reasoning actions planning. The approach is evaluated across eight reasoning tasks, showing consistent outperformance compared to static reasoning techniques and vanilla instruction tuning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper proposes a new way to help large language models reason better. Currently, these models are given instructions on how to solve problems, but this method doesn’t take into account the complexity of the problem or the model’s own abilities. The authors introduce an approach called DOTS that lets the model find its own best way to reason about each question. This is done by breaking down complex reasoning processes into smaller steps and then finding the most effective sequence of these steps for each specific question. The authors tested this approach on eight different problem-solving tasks and found it outperformed other methods. |
Keywords
» Artificial intelligence » Fine tuning » Instruction tuning
