Summary of Interpolated-mlps: Controllable Inductive Bias, by Sean Wu et al.
Interpolated-MLPs: Controllable Inductive Bias
by Sean Wu, Jordan Hong, Keyu Bai, Gregor Bachmann
First submitted to arxiv on: 12 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper investigates the relationship between inductive bias and performance at low compute levels. It proposes an “Interpolated MLP” (I-MLP) approach to control inductive bias, which is achieved by interpolating between fixed weights from a prior model with high inductive bias. The method allows for fractional control of inductive bias, which may be useful when full inductive bias is not desired. Experimental results show that there is a continuous and two-sided logarithmic relationship between inductive bias and performance for Vision Tasks at low compute levels using CNN and MLP-Mixer prior models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper looks at how changing the amount of “hidden knowledge” in a computer model affects its performance when it doesn’t have much computing power. It comes up with a new way to do this, called Interpolated MLP, which lets you control how much hidden knowledge is used. This can be helpful when you don’t want all your computer’s power. The results show that there’s a special pattern between the amount of hidden knowledge and how well the model performs. |
Keywords
» Artificial intelligence » Cnn
