Summary of Graph Of Records: Boosting Retrieval Augmented Generation For Long-context Summarization with Graphs, by Haozhen Zhang et al.
Graph of Records: Boosting Retrieval Augmented Generation for Long-context Summarization with Graphs
by Haozhen Zhang, Tao Feng, Jiaxuan You
First submitted to arxiv on: 14 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed method, graph of records (GoR), enhances retrieval-augmented generation (RAG) for long-context global summarization. GoR leverages historical responses generated by Large Language Models (LLMs) to improve RAG’s performance. Inspired by the retrieve-then-generate paradigm of RAG, GoR constructs a graph by connecting retrieved text chunks with corresponding LLM-generated responses. A graph neural network and a BERTScore-based objective are used for self-supervised model training, enabling supervision signal backpropagation between reference summaries and node embeddings. Experimental results show that GoR outperforms 12 baselines across four long-context summarization datasets, achieving improvements of up to 19% over retrievers on the WCEP dataset. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary GoR is a new way to make language models better at summarizing texts. It takes advantage of information that’s often thrown away, like historical responses generated by large language models. This helps improve the quality of summaries for longer texts. The method uses a special type of graph and neural network to learn from this extra information. Tests showed that GoR is more effective than other methods at summarizing texts. |
Keywords
» Artificial intelligence » Backpropagation » Graph neural network » Neural network » Rag » Retrieval augmented generation » Self supervised » Summarization
