Summary of Galla: Graph Aligned Large Language Models For Improved Source Code Understanding, by Ziyin Zhang et al.
GALLa: Graph Aligned Large Language Models for Improved Source Code Understanding
by Ziyin Zhang, Hang Yu, Shijie Li, Peng Di, Jianguo Li, Rui Wang
First submitted to arxiv on: 6 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper presents a novel approach to incorporating structural information about code into large language models (LLMs). Current code LLMs solely focus on text tokens, ignoring other important aspects like data flow graphs. The authors propose GALLa, a framework that combines graph neural networks and cross-modal alignment techniques to inject structural code information into LLMs during finetuning. This approach is model-agnostic and task-agnostic, allowing it to be applied to any code LLM for various downstream tasks. The authors validate the effectiveness of GALLa on five code tasks with four different baseline LLMs, demonstrating consistent improvements over the baselines. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary Code models can learn a lot from understanding how data flows through programs. Current language models only look at the words in the code, not the way the data moves around. To fix this, researchers created GALLa, which combines special computer vision techniques with big language models. This lets GALLa learn about both the text and the structure of code all at once. The result is a better code model that can do many tasks well, even with really powerful language models like LLaMA3. |
Keywords
» Artificial intelligence » Alignment
