Summary of Rethinking the Effectiveness Of Graph Classification Datasets in Benchmarks For Assessing Gnns, by Zhengdao Li et al.
Rethinking the Effectiveness of Graph Classification Datasets in Benchmarks for Assessing GNNs
by Zhengdao Li, Yong Cao, Kefan Shuai, Yiming Miao, Kai Hwang
First submitted to arxiv on: 6 Jul 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed empirical protocol based on a fair benchmarking framework investigates the performance discrepancy between simple methods and GNNs. To quantify the effectiveness of datasets in distinguishing advancements of GNNs, a novel metric is introduced considering both dataset complexity and model performance. This study thoroughly examines and provides an explicit definition for dataset effectiveness in graph learning, aligning with existing studies and intuitive assumptions across 16 real-world datasets. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper looks at how well graph classification benchmarks work. It seems that simple methods like MLPs can perform just as well as more complex Graph Neural Networks (GNNs). The question is: do these benchmarks actually show what GNNs are good for? To answer this, the authors propose a new way to test and compare different models using fair benchmarking rules. They also introduce a new metric to measure how well a dataset helps or hinders the performance of different models. By testing on 16 real-world datasets, they found that their metric agrees with what other studies have shown and makes sense intuitively. |
Keywords
» Artificial intelligence » Classification
