Summary of Enabling High-sparsity Foundational Llama Models with Efficient Pretraining and Deployment, by Abhinav Agarwalla et al.
Enabling High-Sparsity Foundational Llama Models with Efficient Pretraining and Deployment
by Abhinav Agarwalla, Abhay Gupta, Alexandre Marques, Shubhra Pandit, Michael Goin, Eldar Kurtic, Kevin Leong, Tuan Nguyen, Mahmoud Salem, Dan Alistarh, Sean Lie, Mark Kurtz
First submitted to arxiv on: 6 May 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper introduces a novel approach to create accurate, sparse versions of large language models (LLMs) that achieve full accuracy recovery for fine-tuning tasks at up to 70% sparsity. The method combines the SparseGPT one-shot pruning technique with sparse pretraining on a subset of datasets. The authors demonstrate training acceleration due to sparsity on Cerebras CS-3 chips and inference acceleration of up to 3x on CPUs and 1.7x on GPUs. By utilizing Neural Magic’s DeepSparse engine and nm-vllm engine, the paper shows a total speedup of up to 8.6x for sparse-quantized LLaMA models across diverse tasks, including chat, instruction following, code generation, arithmetic reasoning, and summarization. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about making large language models smaller and faster without losing their ability to work well. The authors have a new way to make these models more efficient by removing some of the parts that aren’t as important. They test this method on different types of tasks like chatting, following instructions, generating code, doing math problems, and summarizing texts. The results show that they can make the models run up to 8.6 times faster without losing their accuracy. |
Keywords
» Artificial intelligence » Fine tuning » Inference » Llama » One shot » Pretraining » Pruning » Summarization
