Summary of Transformer Tricks: Removing Weights For Skipless Transformers, by Nils Graef
Transformer tricks: Removing weights for skipless transformers
by Nils Graef
First submitted to arxiv on: 18 Apr 2024
Categories
- Main: Machine Learning (cs.LG)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The abstract describes a skipless transformer without V and P linear layers, which reduces the total number of weights. This scheme is only applicable to multi-head attention (MHA) but not for multi-query attention (MQA) or grouped-query attention (GQA), which are used by popular large language models (LLMs). The paper proposes mathematically equivalent versions suitable for MQA and GQA, showing that removing Q and P from a skipless version of Mistral-7B would reduce its weights by 15%, compute complexity, and memory requirements. The authors provide code and more transformer tricks in arXiv:2402.13388 and this GitHub URL. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper explains how to make large language models like Llama 2, Mistral, Mixtral, PaLM, and Gemma smaller and faster without losing their abilities. It’s about finding a way to reduce the number of calculations needed by these models while keeping them as good at understanding and generating text. The authors show that this can be done by removing certain parts of the model, making it more efficient and using less computer resources. |
Keywords
» Artificial intelligence » Attention » Llama » Multi head attention » Palm » Transformer
