Summary of Memorization in Deep Learning: a Survey, by Jiaheng Wei et al.
Memorization in deep learning: A survey
by Jiaheng Wei, Yanjun Zhang, Leo Yu Zhang, Ming Ding, Chao Chen, Kok-Leong Ong, Jun Zhang, Yang Xiang
First submitted to arxiv on: 6 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A novel survey delves into the intricate workings of Deep Neural Networks (DNNs), revealing an enigmatic phenomenon where these models tend to memorize specific details rather than learning general patterns. This memorization tendency has significant implications for DNN generalization, security, and privacy. The paper presents a systematic framework to categorize memorization definitions based on generalization and security/privacy domains, summarizes evaluation methods at both example and model levels, and explores the connections between memorization, forgetting, and applications like noisy label learning, privacy preservation, and model enhancement. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Deep Neural Networks are super smart computers that can learn from data. But sometimes they get stuck on tiny details instead of understanding big patterns. This makes it hard for them to work well in new situations or keep our private information safe. A group of researchers has written a report that tries to understand why this happens and how it affects the way we use these super smart computers. |
Keywords
» Artificial intelligence » Generalization
