Summary of Auditing An Automatic Grading Model with Deep Reinforcement Learning, by Aubrey Condor et al.
Auditing an Automatic Grading Model with deep Reinforcement Learning
by Aubrey Condor, Zachary Pardos
First submitted to arxiv on: 11 May 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Emerging Technologies (cs.ET); Machine Learning (cs.LG)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary A reinforcement learning approach is employed to audit an automatic short answer grading (ASAG) model. The ASAG model is designed to match human ratings from a training set, with accuracy metrics used to evaluate its quality. However, this method does not guarantee infallibility of the ASAG model. To address this issue, a reinforcement learning agent is trained to revise student responses aiming for a high rating from the ASAG model in the fewest revisions possible. The agent’s revised responses achieving a high grade but not meeting human standards expose shortcomings in the grading model. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary We’re exploring how an automatic grading system works. Some smart computers are trying to help teachers grade students’ answers, but we need to make sure these computers are doing it correctly. Right now, people are using special tests to see if the computer’s grades match the teacher’s, and that’s not enough. We want to know if the computer is good at grading or just pretending to be. So, we’re training a super smart AI to change student answers to get high marks from the computer, but in ways that wouldn’t make sense to humans. This shows us where the computer is going wrong. |
Keywords
» Artificial intelligence » Reinforcement learning
