Summary of Simplifying the Theory on Over-smoothing, by Andreas Roth
Simplifying the Theory on Over-Smoothing
by Andreas Roth
First submitted to arxiv on: 16 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel paper tackles the complexities of graph convolutions, shedding light on their limitations in inducing over-smoothing effects. By recognizing over-smoothing as a special case of power iteration, researchers can better understand and address this phenomenon. The study presents a comprehensive definition of rank collapse, a generalized form of over-smoothing, along with a corresponding metric, the rank-one distance. Empirically evaluating 14 commonly used methods, the paper reveals that more models suffer from this issue than previously thought. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary A team of researchers has made progress in understanding graph convolutions, which are useful for processing data with irregular structures. One problem with these convolutions is that they can make representations look very similar as they go deeper. This is called over-smoothing. The authors show that over-smoothing is a type of power iteration, making it easier to understand. They also propose a new way to measure this effect and test 14 different methods to see how many are affected. |
