Summary of Difformer: Scalable (graph) Transformers Induced by Energy Constrained Diffusion, By Qitian Wu et al.
DIFFormer: Scalable (Graph) Transformers Induced by Energy Constrained Diffusion
by Qitian Wu, Chenxiao Yang, Wentao Zhao, Yixuan He, David Wipf, Junchi Yan
First submitted to arxiv on: 23 Jan 2023
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The energy-constrained diffusion model introduced by this paper encodes instances from a dataset into evolutionary states that incorporate information from other instances. The model’s diffusion process is constrained by a principled energy function that ensures the consistency of instance representations over latent structures. This leads to closed-form optimal estimates for pairwise diffusion strength, allowing for a new class of neural encoders called DIFFormer (diffusion-based Transformers). Two instantiations are presented: a simple version with linear complexity for large instance numbers and an advanced version for learning complex structures. The model demonstrates superior performance in various tasks such as node classification on graphs, semi-supervised image/text classification, and spatial-temporal dynamics prediction. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper creates a new way to understand data relationships by using a special kind of computer program called a “diffusion model”. This model takes information from many pieces of data and combines it in a smart way to help computers learn about the world. The model is really good at understanding big complex systems, like graphs and images, and can even predict what will happen next. It’s like having a superpower for computers! |
Keywords
* Artificial intelligence * Classification * Diffusion * Diffusion model * Semi supervised * Text classification
