Summary of One Prompt Is Not Enough: Automated Construction Of a Mixture-of-expert Prompts, by Ruochen Wang and Sohyun An and Minhao Cheng and Tianyi Zhou and Sung Ju Hwang and Cho-jui Hsieh
One Prompt is not Enough: Automated Construction of a Mixture-of-Expert Prompts
by Ruochen Wang, Sohyun An, Minhao Cheng, Tianyi Zhou, Sung Ju Hwang, Cho-Jui Hsieh
First submitted to arxiv on: 28 Jun 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A new approach to automating instruction design for Large Language Models (LLMs) is proposed in this paper. By dividing the problem space into sub-regions and assigning specialized experts with demos and instructions, the Mixture-of-Prompts (MoP) method achieves an average win rate of 81% across several benchmarks. This is a significant improvement over prior arts that restricted prompt design to a single instruction. The MoP method consists of two phases: demo assignment, which groups demos into experts based on semantic similarity, and instruction assignment, which searches for the best instruction per expert. This synergy enables LLMs to generalize better to novel tasks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Large Language Models (LLMs) can do many things when given instructions. They’re really good at following rules! To make them even better, researchers wanted to find a way to teach them without giving them explicit instructions. Instead, they used “demos” – short examples of what the LLM should do. The problem was that these demos were hard to design and needed to be connected to the right instruction. In this paper, scientists created a new method called Mixture-of-Prompts (MoP) that solves this problem by breaking down the task into smaller parts and giving each part its own expert with both an instruction and demos. This helps LLMs generalize better to new tasks. |
Keywords
» Artificial intelligence » Prompt
