Summary of Information Plane Analysis Visualization in Deep Learning Via Transfer Entropy, by Adrian Moldovan et al.
Information Plane Analysis Visualization in Deep Learning via Transfer Entropy
by Adrian Moldovan, Angel Cataron, Razvan Andonie
First submitted to arxiv on: 1 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Information Theory (cs.IT)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a novel approach to analyzing the relationship between neural network compression and generalization performance using Transfer Entropy (TE) and Information Plane analysis. The authors argue that TE can capture temporal relationships between variables, unlike mutual information, which is commonly used in this context. They show encouraging experimental results by quantifying information transfer between neural layers and performing Information Plane analysis. This work opens up new possibilities for investigating the causal link between compression and generalization. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper studies how artificial neural networks work and why they’re good at learning things. The authors use a special method called Transfer Entropy to see if there’s a connection between how much information a network stores and its ability to make predictions. They also use a way of visualizing this called Information Plane analysis. The results are promising, suggesting that we can learn more about how networks work by looking at the flow of information between different parts. |
Keywords
» Artificial intelligence » Generalization » Neural network
