Summary of Near: a Training-free Pre-estimator Of Machine Learning Model Performance, by Raphael T. Husistein et al.
NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance
by Raphael T. Husistein, Markus Reiher, Marco Eckhoff
First submitted to arxiv on: 16 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Disordered Systems and Neural Networks (cond-mat.dis-nn); Chemical Physics (physics.chem-ph); Data Analysis, Statistics and Probability (physics.data-an)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a zero-cost proxy for Neural Architecture Search (NAS) called Network Expressivity by Activation Rank (NEAR). NEAR is based on the effective rank of pre- and post-activation matrices in neural networks, which correlates well with model accuracy. The authors demonstrate this correlation on NAS-Bench-101 and NATS-Bench-SSS/TSS datasets. Additionally, they present a simple approach to estimate optimal layer sizes in multi-layer perceptrons (MLPs). Furthermore, NEAR can be used to select hyperparameters such as activation functions and weight initialization schemes. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us build better artificial intelligence models by finding the right combination of building blocks without needing lots of computer power. They came up with a new way to score neural networks based on how well their layers work together, which is closely linked to how accurate the model will be. They tested this idea on several famous datasets and showed that it works well. This could help us make AI models more efficient and easier to create. |
