Summary of Atp: Enabling Fast Llm Serving Via Attention on Top Principal Keys, by Yue Niu et al.
ATP: Enabling Fast LLM Serving via Attention on Top Principal Keys
by Yue Niu, Saurav Prakash, Salman Avestimehr
First submitted to arxiv on: 1 Mar 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed Attention mechanism with Linear Complexity (ATP) addresses the quadratic complexity of traditional attention mechanisms by focusing on top principal keys. ATP is driven by the observation that input sequences are typically low-rank, allowing for efficient computation by transforming inputs into an orthogonal space and attending only to the top principal bases. This reduction in complexity translates to a performance drop of only 2% when using 1/4 principal keys. ATP achieves comparable accuracy with lower computation and memory complexity than standard attention mechanisms on various models such as BERT and Llama. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary ATP is a new way to look at how our computers understand what we’re saying. It’s like having a special filter that helps them focus on the most important parts of what we’re talking about. Right now, computers use a lot of energy trying to understand everything we say, which can slow them down. ATP helps solve this problem by looking only at the most important parts and ignoring the rest. This makes it faster and more efficient! It works just as well on big models like BERT and Llama. |
Keywords
* Artificial intelligence * Attention * Bert * Llama
