Summary of Towards Neural Scaling Laws on Graphs, by Jingzhe Liu et al.
Towards Neural Scaling Laws on Graphs
by Jingzhe Liu, Haitao Mao, Zhikai Chen, Tong Zhao, Neil Shah, Jiliang Tang
First submitted to arxiv on: 3 Feb 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The abstract presents a study that investigates the neural scaling laws on graphs, examining how deep graph models change in performance with model and dataset sizes. The researchers benchmark various graph datasets from different tasks to establish these laws from both model and data perspectives. They find that model size is not the only factor affecting scaling behaviors, as model depth also plays a significant role. For data scaling, they propose reforming the law using node or edge counts instead of graph count, due to irregular graph sizes. The study provides valuable insights into neural scaling laws on graphs, which can inform collecting new graph data and developing large graph models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Deep learning models are used to analyze different types of graphs. These models help share knowledge across various graphs. However, scientists haven’t fully understood how these models work as the number of model parameters and the size of the dataset increase. The researchers in this study looked at many datasets and tried to figure out how deep graph models change when the model gets bigger or the dataset grows. They found that the depth of the model is important too, not just the number of parameters. For data, they realized that counting the number of graphs isn’t helpful because graphs come in different sizes. Instead, they used the number of nodes or edges to understand how the data changes when it gets bigger. This study helps us better understand deep learning models on graphs and can be useful for creating new graph data and large models. |
Keywords
* Artificial intelligence * Deep learning * Scaling laws
