Summary of Chess: Optimizing Llm Inference Via Channel-wise Thresholding and Selective Sparsification, by Junhui He et al.
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification
by Junhui He, Shangyu Wu, Weidong Wen, Chun Jason Xue, Qingan Li
First submitted to arxiv on: 2 Sep 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a novel approach to deploying large language models on edge devices, addressing the challenges of computational overhead and memory requirements. The authors reformulate the activation sparsification problem to explicitly capture the relationship between activation sparsity and model performance. They introduce CHESS, a general activation sparsification method that combines channel-wise thresholding and selective sparsification. This approach achieves lower performance degradation over eight downstream tasks while activating fewer parameters than existing methods, resulting in up to 1.27x speedup during inference. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps make big language models work better on devices like smartphones. Right now, these models are too big for those devices because they need lots of computing power and memory. The authors found a way to shrink the models without making them perform worse. They created a new method that works by setting different rules for each part of the model, and then choosing which parts to reduce. This makes the models work faster on edge devices, like up to 1.27 times faster. |
Keywords
» Artificial intelligence » Inference
