Summary of Half-space Feature Learning in Neural Networks, by Mahesh Lorik Yadav et al.
Half-Space Feature Learning in Neural Networks
by Mahesh Lorik Yadav, Harish Guruprasad Ramaswamy, Chandrashekar Lakshminarayanan
First submitted to arxiv on: 5 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes an alternative perspective on neural network feature learning, arguing that neither extreme viewpoint (NTK-like kernel method or intricate hierarchical features) is accurate. Instead, it views neural networks as a mixture of experts, where each expert corresponds to a path through hidden units. This perspective motivates the Deep Linearly Gated Network (DLGN), which combines linear and non-linear feature learning capabilities. The DLGN’s architecture allows for global visualization of features, unlike local visualizations based on saliency/activation/gradient maps. The paper shows that feature learning in DLGNS occurs through learning half-spaces in the input space that contain smooth regions of the target function. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Neural networks can be super confusing! But imagine them like a bunch of experts working together to learn from data. This paper takes a fresh look at how neural networks learn features and suggests that they’re not just simple or super complex, but something in between. They propose an “expert” model called the Deep Linearly Gated Network (DLGN) that combines simple and non-simple feature learning. It’s like having a map to see where all the features are hiding! |
Keywords
* Artificial intelligence * Mixture of experts * Neural network
