Summary of Understand What Llm Needs: Dual Preference Alignment For Retrieval-augmented Generation, by Guanting Dong et al.
Understand What LLM Needs: Dual Preference Alignment for Retrieval-Augmented Generation
by Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Zhicheng Dou, Ji-Rong Wen
First submitted to arxiv on: 26 Jun 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary DPA-RAG, a universal framework, aims to address the challenge of aligning diverse knowledge preferences within retrieval-augmented generation (RAG) systems. To achieve this, it introduces a preference knowledge construction pipeline and five novel query augmentation strategies to alleviate preference data scarcity. The framework accomplishes both external and internal preference alignment: joint pair-wise, point-wise, and contrastive preference alignment in the reranker for external alignment, and pre-aligned stage before vanilla Supervised Fine-tuning (SFT) for internal alignment. Experimental results on four knowledge-intensive QA datasets demonstrate that DPA-RAG outperforms all baselines and seamlessly integrates both black-box and open-sourced LLM readers. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary DPA-RAG is a new way to make large language models work better together. It helps them understand what they are looking for in the same way, so they can give more accurate answers. The system has two parts: one that makes sure all the different models are working with the same goals, and another that helps the models learn from each other’s strengths and weaknesses. |
Keywords
* Artificial intelligence * Alignment * Fine tuning * Rag * Retrieval augmented generation * Supervised
