Summary of Abstraction Requires Breadth: a Renormalisation Group Approach, by Carlo Orientale Caputo et al.
Abstraction requires breadth: a renormalisation group approach
by Carlo Orientale Caputo, Elias Seiffert, Matteo Marsili
First submitted to arxiv on: 1 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Disordered Systems and Neural Networks (cond-mat.dis-nn); Data Analysis, Statistics and Probability (physics.data-an); Neurons and Cognition (q-bio.NC); Machine Learning (stat.ML)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper explores the concept of abstraction in machine learning, drawing parallels between data processing and statistical physics. Researchers have observed that neural networks develop abstract characteristics, such as “cat-ness” or “dog-ness,” through deep layers combining lower-level features. However, they argue that depth alone is insufficient to produce truly abstract representations. Instead, the level of abstraction crucially depends on the breadth of the training set. The paper proposes a Hierarchical Feature Model, a theoretical representation that can be tested using Deep Belief Networks trained on data of varying breadth. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at how computers learn from big datasets and create general ideas or patterns. It compares this process to something called renormalisation group in physics, where they simplify complex systems by removing small details. In machine learning, researchers found that deeper neural networks can pick up on bigger patterns in data, like what makes a picture of a cat a cat. But the study says it’s not just how deep the network is, but also how much data it sees that matters. The scientists are trying to figure out what they call an “abstract representation,” or a way to describe things without getting into too many details. They’re testing this idea using special computer models and training them on different types of data. |
Keywords
» Artificial intelligence » Machine learning
