Summary of Applying Irt to Distinguish Between Human and Generative Ai Responses to Multiple-choice Assessments, by Alona Strugatski and Giora Alexandron
Applying IRT to Distinguish Between Human and Generative AI Responses to Multiple-Choice Assessments
by Alona Strugatski, Giora Alexandron
First submitted to arxiv on: 28 Nov 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: None
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The proposed method uses Item Response Theory (IRT) to detect AI cheating in multiple-choice questions (MCQs). It assumes that AI-generated responses exhibit different patterns compared to human responses. The approach models these deviations using Person-Fit Statistics, demonstrating its effectiveness in distinguishing between human and chatbot-generated responses from leading AI models like ChatGPT, Claude, and Gemini. Additionally, the study reveals differences in reasoning profiles among the chatbots. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary This paper is about finding ways to prevent students from cheating with the help of artificial intelligence (AI). Right now, there’s not much research on how to spot AI cheating in multiple-choice tests. The researchers came up with a new method that uses a type of statistics called Item Response Theory (IRT) to figure out if answers are coming from humans or AI. They tested their approach using famous chatbots like ChatGPT and found it worked pretty well. |
Keywords
» Artificial intelligence » Claude » Gemini
