Summary of Controlling Cloze-test Question Item Difficulty with Plm-based Surrogate Models For Irt Assessment, by Jingshen Zhang and Jiajun Xie and Xinying Qiu
Controlling Cloze-test Question Item Difficulty with PLM-based Surrogate Models for IRT Assessment
by Jingshen Zhang, Jiajun Xie, Xinying Qiu
First submitted to arxiv on: 3 Mar 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The authors introduce a novel approach for generating multiple-choice (MC) cloze test questions with varying levels of difficulty, leveraging pre-trained language models (PLMs) as surrogate models. This enables item response theory (IRT) assessment without requiring human test subjects. The framework employs two strategies to control the difficulty levels of both gaps and distractors using ranking rules to minimize invalid distractors. Experimentation on a benchmark dataset demonstrates the effectiveness of this approach in controlling and evaluating the difficulty levels of MC cloze tests. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper creates new multiple-choice questions that are easy or hard, depending on what you want. It uses special computers (language models) to make the questions instead of asking people to help. This way, we can test how well someone knows something without bothering anyone. The authors also figure out ways to make sure the wrong answers aren’t too obvious, so it’s not easy to pick just one correct answer. They tested this on a big dataset and it worked pretty well! |
