Summary of Adaptive Pruning For Large Language Models with Structural Importance Awareness, by Haotian Zheng et al.
Adaptive Pruning for Large Language Models with Structural Importance Awareness
by Haotian Zheng, Jinke Ren, Yushan Sun, Ruichen Zhang, Wenbo Zhang, Zhen Li, Dusit Niyato, Shuguang Cui, Yatong Han
First submitted to arxiv on: 19 Dec 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel method for pruning large language models (LLMs) to reduce their computational and storage demands while maintaining performance. The approach, called structurally-aware adaptive pruning (SAAP), uses an adaptive importance fusion metric to evaluate the importance of all coupled structures in LLMs. This allows for targeted pruning of specific layers to meet performance requirements. The paper also develops a group fine-tuning strategy to improve inference efficiency. Experimental results show that SAAP outperforms state-of-the-art methods on multiple LLMs, achieving 2.17%, 2.37%, and 2.39% accuracy gains on LLaMA-7B, Vicuna-7B, and LLaMA-13B. Additionally, SAAP improves token generation speed by 5%. The proposed method has practical advantages in resource-constrained scenarios. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making language models more efficient so they can work on devices with limited resources. Currently, these models are very big and require a lot of computer power and storage space. To fix this problem, the researchers developed a new way to reduce the size of these models while keeping their performance the same. They used special math to figure out which parts of the model are most important and removed the less important parts. This made the model smaller and faster, but still worked just as well. The researchers tested their method on several language models and found that it works better than other methods. This is important because it means we can use these models in more places, like smart devices or robots. |
Keywords
» Artificial intelligence » Fine tuning » Inference » Llama » Pruning » Token
