Summary of A Llm-powered Automatic Grading Framework with Human-level Guidelines Optimization, by Yucheng Chu et al.
A LLM-Powered Automatic Grading Framework with Human-Level Guidelines Optimization
by Yucheng Chu, Hang Li, Kaiqi Yang, Harry Shomer, Hui Liu, Yasemin Copur-Gencturk, Jiliang Tang
First submitted to arxiv on: 3 Oct 2024
Categories
- Main: Artificial Intelligence (cs.AI)
- Secondary: Computation and Language (cs.CL)
GrooveSquid.com Paper Summaries
GrooveSquid.com’s goal is to make artificial intelligence research accessible by summarizing AI papers in simpler terms. Each summary below covers the same AI paper, written at different levels of difficulty. The medium difficulty and low difficulty versions are original summaries written by GrooveSquid.com, while the high difficulty version is the paper’s original abstract. Feel free to learn from the version that suits you best!
| Summary difficulty | Written by | Summary |
|---|---|---|
| High | Paper authors | High Difficulty Summary Read the original abstract here |
| Medium | GrooveSquid.com (original content) | Medium Difficulty Summary The paper proposes a unified multi-agent automatic short-answer grading (ASAG) framework called GradeOpt that leverages large language models (LLMs) for grading open-ended short-answer questions. The framework incorporates two additional LLM-based agents, the reflector and refiner, which enable self-reflection on errors to optimize original grading guidelines. In experiments, GradeOpt demonstrates superior performance in grading accuracy and behavior alignment with human graders compared to representative baselines. |
| Low | GrooveSquid.com (original content) | Low Difficulty Summary The paper is about a new way to grade open-ended questions that asks students for their thoughts or ideas. This method can help teachers know what students really understand, but it takes a lot of time to grade all the answers by hand. The new system uses big language models to do the grading instead, and it even learns from its mistakes to get better at grading. The results show that this new system is much better than other methods at getting the right answers and doing things in a way that’s similar to how human teachers would grade. |
Keywords
» Artificial intelligence » Alignment
