Summary of Human-like Episodic Memory For Infinite Context Llms, by Zafeirios Fountas et al.
Human-like Episodic Memory for Infinite Context LLMs
by Zafeirios Fountas, Martin A Benfeghoul, Adnan Oomerjee, Fenia Christopoulou, Gerasimos Lampouras, Haitham Bou-Ammar, Jun Wang
First submitted to arxiv on: 12 Jul 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL); Machine Learning (cs.LG); Neurons and Cognition (q-bio.NC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper introduces EM-LLM, a novel approach that integrates key aspects of human episodic memory and event cognition into large language models (LLMs). This allows LLMs to handle practically infinite context lengths while maintaining computational efficiency. The approach organises sequences of tokens into coherent episodic events using Bayesian surprise and graph-theoretic boundary refinement. It also retrieves these events through a two-stage memory process, combining similarity-based and temporally contiguous retrieval for efficient access to relevant information. The paper demonstrates EM-LLM’s superior performance on the LongBench and InfiniteBench benchmarks, outperforming state-of-the-art models InfLLM and RAG in various tasks while requiring similar resources. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary EM-LLM is a new way to make language models better at understanding long texts. It works by organizing words into events that happened at specific times, just like our brains do when we remember things that happened to us. This helps the model keep track of lots of information without getting confused or losing its train of thought. The authors tested EM-LLM and found it did much better than other models on big tasks. |
Keywords
* Artificial intelligence * Rag
