Summary of Reinforcement Learning For Question Answering in Programming Domain Using Public Community Scoring As a Human Feedback, by Alexey Gorbatovski and Sergey Kovalchuk
Reinforcement learning for question answering in programming domain using public community scoring as a human feedback
by Alexey Gorbatovski, Sergey Kovalchuk
First submitted to arxiv on: 19 Jan 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper investigates how to improve the performance of the GPT Neo 125M language model in answering programming-related questions on community forums. The researchers integrate Reinforcement Learning from Human Feedback (RLHF) with scores from Stack Overflow and employ two distinct training strategies using Proximal Policy Optimization (PPO). They find that this method achieves comparable improvements to a larger parameter variant, GPT Neo 2.7B. Additionally, the study highlights the limitations of traditional linguistic metrics in evaluating responses and introduces an auxiliary scoring mechanism to address this issue. The paper underscores the importance of domain-specific evaluation methods for refining Large Language Models through human feedback. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This study looks at how to make a language model better at answering programming questions online. The researchers use two new ways to train the model and test it on programming-related questions. They find that this approach works almost as well as a larger version of the same model. The study also shows why traditional methods for measuring how good an answer is might not work well for programming questions. It highlights the need for special evaluation methods that take into account what makes a good answer in the programming domain. |
Keywords
» Artificial intelligence » Gpt » Language model » Optimization » Reinforcement learning from human feedback » Rlhf
