Summary of Sloth: Scaling Laws For Llm Skills to Predict Multi-benchmark Performance Across Families, by Felipe Maia Polo et al.
Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families
by Felipe Maia Polo, Seamus Somerstep, Leshem Choshen, Yuekai Sun, Mikhail Yurochkin
First submitted to arxiv on: 9 Dec 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces Skills Scaling Laws (SSLaws), a novel approach to predict large language model (LLM) performance. Unlike traditional scaling laws, SSLaws leverages publicly available benchmark data to capture the relationships between LLM performance, model size, and training data across different model families. The authors demonstrate that this approach provides more accurate and interpretable predictions compared to existing methods, without requiring the training of multiple models per family. The proposed method is evaluated on 12 prominent benchmarks from Open LLM Leaderboard v1/v2, showing improved predictive performance for complex downstream tasks and increased test-time compute. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This research focuses on improving our understanding of how large language models (LLMs) work. Traditional methods try to predict an LLM’s performance based on its size and training data. However, this approach has limitations because different LLMs are trained in slightly different ways. The authors propose a new method called Skills Scaling Laws that takes into account the variations between different LLMs and how they perform on various tasks. This method is tested on many benchmarks and shows better results than existing methods. |
Keywords
» Artificial intelligence » Large language model » Scaling laws
