Summary of The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models, by Adithya Bhaskar et al.
The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models
by Adithya Bhaskar, Dan Friedman, Danqi Chen
First submitted to arxiv on: 6 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Medium Difficulty Summary: This paper explores the phenomenon of syntactic generalization in language models (LMs), where fine-tuned models with different random seeds achieve similar performance but differ in their ability to generalize. The authors investigate whether this can be attributed to competing subnetworks within a single model, as previously suggested for simpler algorithmic tasks (“grokking”). Instead, they find that multiple subnetworks share a set of attention heads, referred to as the heuristic core, which emerge early in training and compute shallow features. The model generalizes by incorporating additional attention heads that depend on the outputs of these heuristic heads to compute higher-level features. This study provides a more detailed understanding of syntactic generalization mechanisms in pretrained LMs. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Low Difficulty Summary: Imagine you have a special kind of computer program called a language model (LM) that can learn from lots of text data. When you fine-tune this program to understand a specific topic, it might perform well but still struggle with understanding certain rules or patterns in language. This paper tries to figure out why this happens and how the program’s internal mechanisms change as it learns. Surprisingly, they found that different parts of the program (called subnetworks) are actually working together in harmony, rather than competing against each other. These subnetworks share a set of “rules” or attention heads that help the model learn to recognize patterns and eventually generalize better. |
Keywords
* Artificial intelligence * Attention * Generalization * Language model
