Summary of Increasing the Difficulty Of Automatically Generated Questions Via Reinforcement Learning with Synthetic Preference, by William Thorne et al.
Increasing the Difficulty of Automatically Generated Questions via Reinforcement Learning with Synthetic Preference
by William Thorne, Ambrose Robinson, Bohua Peng, Chenghua Lin, Diana Maynard
First submitted to arxiv on: 10 Oct 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This paper proposes a cost-effective approach to generating domain-specific Machine Reading Comprehension (MRC) datasets for cultural heritage information using Reinforcement Learning from Human Feedback (RLHF). The authors target the final answering task in MRC, which is well-suited for cultural heritage applications. To achieve this, they develop a methodology that leverages existing question-answering models on a subset of SQuAD to create a difficulty metric. This method increases question difficulty using Proximal Policy Optimization (PPO) and synthetic data. The authors provide empirical evidence of the approach’s effectiveness through human evaluation, error analysis, and a study of emergent phenomena. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper helps us better understand how to make machines learn from humans. It’s like teaching a machine to find answers in old books or museum collections. Right now, making these datasets by hand is too expensive for most museums. So, the researchers created a way to generate these datasets more efficiently using computer algorithms and synthetic data. They tested their method on existing question-answering models and found it works well. This can help museums and libraries make their collections more accessible online. |
Keywords
» Artificial intelligence » Optimization » Question answering » Reinforcement learning from human feedback » Rlhf » Synthetic data
