Summary of Optima: Optimizing Effectiveness and Efficiency For Llm-based Multi-agent System, by Weize Chen et al.
Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System
by Weize Chen, Jiarui Yuan, Chen Qian, Cheng Yang, Zhiyuan Liu, Maosong Sun
First submitted to arxiv on: 10 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel framework called Optima is presented to enhance the performance of Large Language Model (LLM) based multi-agent systems (MAS) in collaborative problem-solving. The current limitations of LLM-based MAS, including low communication efficiency, poor scalability, and a lack of effective parameter-updating optimization methods, are addressed through an iterative generate, rank, select, and train paradigm with a reward function balancing task performance, token efficiency, and communication readability. The effectiveness-efficiency trade-offs of various reinforcement learning (RL) algorithms, including Supervised Fine-Tuning, Direct Preference Optimization, and their hybrid approaches, are explored. Monte Carlo Tree Search-inspired techniques are integrated for data generation, treating conversation turns as tree nodes to explore diverse interaction paths. Optima is evaluated on common multi-agent tasks, including information-asymmetric question answering and complex reasoning, showing consistent and substantial improvements over single-agent baselines and vanilla MAS based on Llama 3 8B. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Optima is a new way for computers to work together to solve problems. Right now, these computer systems are not very good at communicating with each other or working together efficiently. Optima fixes this by making sure the computers communicate in a way that makes sense and helps them work better together. It also finds the best ways for the computers to learn from each other and get better over time. This new system can help computers solve problems that are too hard for one computer alone, like answering tricky questions or doing complex math. |
Keywords
» Artificial intelligence » Fine tuning » Large language model » Llama » Optimization » Question answering » Reinforcement learning » Supervised » Token
