Summary of Dive: Subgraph Disagreement For Graph Out-of-distribution Generalization, by Xin Sun et al.
DIVE: Subgraph Disagreement for Graph Out-of-Distribution Generalization
by Xin Sun, Liang Wang, Qiang Liu, Shu Wu, Zilei Wang, Liang Wang
First submitted to arxiv on: 8 Aug 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper tackles the challenge of out-of-distribution (OOD) generalization in graph machine learning. Traditional algorithms based on uniform distribution assumptions falter in real-world scenarios where this assumption fails, leading to suboptimal performance. Neural networks trained through Stochastic Gradient Descent (SGD) have an inherent simplicity bias that prefers simpler features over more complex yet predictive ones, resulting in reliance on spurious correlations and poor OOD performance in tasks like image recognition, natural language understanding, and graph classification. Current methodologies, including subgraph-mixup and information bottleneck approaches, have achieved partial success but struggle to overcome simplicity bias. To address this, the authors propose DIVE, training a collection of models to focus on all label-predictive subgraphs by encouraging divergence on the subgraph mask. A regularizer is used to punish overlap in extracted subgraphs across models, promoting distinct structural patterns. Model selection for robust OOD performance is achieved through validation accuracy. The approach demonstrates significant improvement over existing methods, effectively addressing simplicity bias and enhancing generalization in graph machine learning. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper tries to fix a problem in computer science called out-of-distribution (OOD) generalization. Right now, when we train machines to do tasks on graphs, they don’t work well if the data is different from what they were trained on. This is because machines tend to focus on simple patterns instead of more complex ones that might be important for the task. The paper proposes a new way to train machines called DIVE, which helps them find more complex patterns and do better in situations where the data is different from what they were trained on. |
Keywords
» Artificial intelligence » Classification » Generalization » Language understanding » Machine learning » Mask » Stochastic gradient descent
