Summary of Reinforcement Learning For Optimizing Rag For Domain Chatbots, by Mandar Kulkarni et al.
Reinforcement Learning for Optimizing RAG for Domain Chatbots
by Mandar Kulkarni, Praveen Tangarajan, Kyung Kim, Anusua Trivedi
First submitted to arxiv on: 10 Jan 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper presents a Retrieval Augmented Generation (RAG) approach for building a chatbot that answers user queries using Frequently Asked Questions (FAQ) data. The authors train an in-house retrieval embedding model using infoNCE loss, which outperforms a well-known general-purpose public embedding model in terms of retrieval accuracy and Out-of-Domain query detection. The chatbot utilizes an open API-based paid ChatGPT model as the Large Language Model (LLM). To optimize the number of LLM tokens and cost, the authors propose a policy-based model external to the RAG pipeline, which interacts with the RAG pipeline through policy actions and updates the policy to minimize the cost. This approach is demonstrated for a FAQ chatbot but can be applied to any existing RAG pipeline. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper makes it possible to build a helpful chatbot that answers questions using information from Frequently Asked Questions (FAQs). The chatbot uses a big language model, like ChatGPT, and a special technique called Retrieval Augmented Generation. This allows the chatbot to answer follow-up questions more accurately. To make this work efficiently, the authors came up with a way to use Reinforcement Learning to decide when to fetch FAQ context or skip it. They tested this approach with different models and found that it can save costs while still providing good answers. |
Keywords
» Artificial intelligence » Embedding » Language model » Large language model » Rag » Reinforcement learning » Retrieval augmented generation
