Summary of How Many Classifiers Do We Need?, by Hyunsuk Kim et al.
How many classifiers do we need?
by Hyunsuk Kim, Liam Hodgkinson, Ryan Theisen, Michael W. Mahoney
First submitted to arxiv on: 1 Nov 2024
Categories
- Main: Machine Learning (stat.ML)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A new paper delves into the realm of ensembling, where multiple models are combined to improve accuracy, as scaling data and/or model size no longer yields significant performance gains. The study focuses on understanding how classifier disagreement and polarization relate to the effectiveness of majority vote strategies in classification tasks. Researchers derive an upper bound for polarization and propose a neural polarization law, which states that most interpolating neural network models are 4/3-polarized. Empirical results support this conjecture and show that polarization is relatively constant across datasets, regardless of hyperparameters or architectures. The study also explores the error of majority vote classifiers under restricted entropy conditions, presenting a tight upper bound indicating a linear correlation between disagreement and target values. Additionally, the authors prove theoretical results on the asymptotic behavior of disagreement in terms of the number of classifiers, which can aid in predicting performance for larger numbers of classifiers. Experiments are conducted on image classification tasks using various types of neural networks. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Ensembling is a way to make many models work together better. This helps when making predictions becomes harder because there’s too much data or our models get really big. A new study looks at how the disagreements between different models and their “polarization” (a new idea introduced in this paper) affect how well ensembling works. The researchers found some rules that help predict how good ensembling will be, based on how the models disagree with each other and how “polarized” they are. They also did experiments to test these ideas using images and different types of neural networks. |
Keywords
» Artificial intelligence » Classification » Image classification » Neural network
