Summary of Planrag: a Plan-then-retrieval Augmented Generation For Generative Large Language Models As Decision Makers, by Myeonghwa Lee et al.
PlanRAG: A Plan-then-Retrieval Augmented Generation for Generative Large Language Models as Decision Makers
by Myeonghwa Lee, Seonho An, Min-Soo Kim
First submitted to arxiv on: 18 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel approach to decision-making using Large Language Models (LLMs). The authors define Decision QA as the task of identifying the best decision, d_best, for a given question Q, business rules R, and database D. Since there is no existing benchmark for this task, the authors propose the Decision QA Benchmark (DQA), comprising two scenarios: Locating and Building. These scenarios are constructed from video games Europa Universalis IV and Victoria 3, which share similarities with Decision QA. To address Decision QA effectively, the authors introduce a new RAG technique called iterative plan-then-retrieval augmented generation (PlanRAG). The PlanRAG-based LM first generates a plan for decision-making, followed by generating queries for data analysis. This approach outperforms state-of-the-art iterative RAG methods by 15.8% and 7.4% in the Locating and Building scenarios, respectively. The authors release their code and benchmark at this URL. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper explores how computers can help make better decisions by analyzing complex data. They define a new problem called Decision QA, which is like playing video games but instead of characters, it’s about finding the best decision. Since there isn’t a standard way to test this type of decision-making, they created their own benchmark using two popular video games. To solve this problem, they developed a new method called PlanRAG, which first figures out what steps to take and then finds the right data to analyze. This approach did better than previous methods in testing, with improvements of 15.8% and 7.4%. The authors are sharing their code and benchmark so others can use it too. |
Keywords
» Artificial intelligence » Rag » Retrieval augmented generation
