Summary of A Study on the Vulnerability Of Test Questions Against Chatgpt-based Cheating, by Shanker Ram and Chen Qian
A Study on the Vulnerability of Test Questions against ChatGPT-based Cheating
by Shanker Ram, Chen Qian
First submitted to arxiv on: 21 Feb 2024
Categories
- Main: Computation and Language (cs.CL)
- Secondary: Artificial Intelligence (cs.AI); Computers and Society (cs.CY)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary This research paper explores the capabilities of ChatGPT, a chatbot that answers text prompts with impressive accuracy, including on postgraduate-level questions. Educators have raised concerns about students using tools like ChatGPT to cheat on remote tests and exams. To address this issue, the authors investigate how well ChatGPT can answer test questions and develop methods to detect vulnerable questions. They generate responses for over 10,000 medical school entrance exam questions in the MedMCQA dataset and analyze the results. The study finds that certain types of questions are answered more inaccurately than others by ChatGPT. Moreover, the authors create a basic natural language processing model to identify the most susceptible questions in a collection or sample exam. This tool can help test-makers design exams that are less vulnerable to ChatGPT-based cheating. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary ChatGPT is a super smart computer program that answers questions really well. Some teachers worry that students might use this tool to cheat on tests and exams. To solve this problem, scientists looked at how well ChatGPT can answer test questions. They used a huge dataset of medical school entrance exam questions and found out which types of questions ChatGPT gets wrong the most. The researchers also created a simple computer program to find the most vulnerable questions in a test or exam. This tool can help teachers make sure their tests aren’t too easy for students to cheat on. |
Keywords
» Artificial intelligence » Natural language processing
