Summary of In-context Learning Of Physical Properties: Few-shot Adaptation to Out-of-distribution Molecular Graphs, by Grzegorz Kaszuba et al.
In-Context Learning of Physical Properties: Few-Shot Adaptation to Out-of-Distribution Molecular Graphs
by Grzegorz Kaszuba, Amirhossein D. Naghdi, Dario Massa, Stefanos Papanikolaou, Andrzej Jaszkiewicz, Piotr Sankowski
First submitted to arxiv on: 3 Jun 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Materials Science (cond-mat.mtrl-sci)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper explores the potential of large language models to predict material properties outside their training data by leveraging in-context learning. The researchers investigate whether they can utilize this ability to predict out-of-distribution materials properties and propose a compound model that combines GPT-2 with geometry-aware graph neural networks to adapt in-context information. They partition the QM9 dataset into sequences of molecules sharing common substructures for in-context learning, achieving better performance on out-of-distribution examples compared to general graph neural network models. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper shows how big language models can learn new things just by being given a few examples. The researchers want to know if they can use this ability to predict properties of materials that are not in their training data. They create a special model that combines two different types of networks: GPT-2 and geometry-aware graph neural networks. This allows the model to adapt to new information while making predictions. To test their approach, they split a dataset called QM9 into smaller groups of molecules with similar structures and use these for learning. The results show that this method works better than usual models. |
Keywords
* Artificial intelligence * Gpt * Graph neural network
