Summary of Bonsai: Gradient-free Graph Distillation For Node Classification, by Mridul Gupta and Samyak Jain and Vansh Ramani and Hariprasad Kodamana and Sayan Ranu
Bonsai: Gradient-free Graph Distillation for Node Classification
by Mridul Gupta, Samyak Jain, Vansh Ramani, Hariprasad Kodamana, Sayan Ranu
First submitted to arxiv on: 23 Oct 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary Graph distillation has shown promise for training Graph Neural Networks (GNNs) by compressing datasets while preserving essential graph characteristics. Our study reveals significant shortcomings in current graph distillation techniques. These methods typically require full dataset training and fresh distillation for hyperparameter or GNN architecture changes, limiting flexibility and reusability. Moreover, they fail to achieve substantial size reduction due to synthesizing fully-connected graphs. To address these challenges, we introduce Bonsai, a novel method that encodes exemplar computation trees to maximize representation of all trees in the training set. This approach enables linear-time, model-agnostic graph distillation for node classification, outperforming existing baselines across 6 real-world datasets on accuracy while being 22 times faster on average. Our rigorous mathematical guarantees make Bonsai robust to GNN architectures, datasets, and parameters. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Researchers have been trying to find a way to teach computers about complex graphs using less data. They call this “graph distillation”. But there are some big problems with the methods they’re using now. These methods need all the data to work, and if you change any settings or try a new approach, it needs to start over again from scratch. This makes them not very useful for real-world applications. To fix these issues, we created a new method called Bonsai. It works by selecting important “exemplar” patterns in the data that can be used to represent all the other patterns. This allows us to compress the data while still keeping it accurate and useful. Our new method is not only better than the old methods but also much faster, making it a big step forward for computers learning about complex graphs. |
Keywords
» Artificial intelligence » Classification » Distillation » Gnn » Hyperparameter
