Summary of Infllm: Training-free Long-context Extrapolation For Llms with An Efficient Context Memory, by Chaojun Xiao et al.
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory
by Chaojun Xiao, Pengle Zhang, Xu Han, Guangxuan Xiao, Yankai Lin, Zhengyan Zhang, Zhiyuan Liu, Maosong Sun
First submitted to arxiv on: 7 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a novel approach to processing extremely long sequences with large language models (LLMs). Existing LLMs, pre-trained on shorter sequences, struggle to understand longer inputs due to out-of-domain and distraction issues. To address this limitation, the authors introduce InfLLM, a training-free memory-based method that stores distant contexts in additional memory units. This allows LLMs to efficiently process long sequences with limited context windows while capturing long-distance dependencies. Without any fine-tuning, InfLLM enables pre-trained LLMs to achieve comparable performance to competitive baselines trained on longer sequences. The authors demonstrate the effectiveness of InfLLM by processing sequences up to 1,024K tokens. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps large language models understand really long texts without needing to learn all over again. Right now, these models can only handle short texts because they get confused with longer ones. To fix this, the researchers created a new way to store information about distant parts of the text in special “memory units.” This lets the model focus on the most important parts of the long text and still understand it well. The new method works without needing to retrain the model, which is useful because training these models takes a lot of time and computer power. |
Keywords
* Artificial intelligence * Fine tuning
